Xash3D FWGS Engine

(continuation, part 1 is here, also, please forgive if I'm m[ei]ssing something - I'm half asleep)

Rough start

Between me calling the A Team FWGS team and them actually arriving, I kindly asked a1batross to gather all fixes and patches he may have written to this point, and quickly started reviewing and incorporating them into my main production branch. That preparation saved us a lot of time. It was really worth for what was coming.

Before even compiling the engine, we had to install the compiler, debugger and lots of development packages like common headers, SDL and Mesa. a1ba and mittorn did a great job figuring that out - they're very experienced in Linux topic. By the time they finished I have completed new Makefiles (these are... like project files for Visual Studio, but, errm... not project files, and not for VS).

First attempt to actually build XDM was a bit of a mess: from syscalls and debug-printf()s to assembly inlines there were minor bugs lurking here and there. But that was relatively easily fixed with the help of FWGS team's previous experience. And finally - it started! (but didn't go too far...)

First run revealed many positive things, for example:

• Xash3D (FWGS ver.) is starting, not crashing immediately
• title music is playing, sounding a bit odd though
• menus are showing and not lagging
• 3D is also working, OpenGL and Mesa win

But the joy "3D" didn't last long. Game crashed soon after the map got initialized. Turned out, it was an odd, but known to FWGS engine printf() problem. Fixed and recompiled immediately. Re-run and - bang! - killed. In game. This time by monsters. :)

So, we're getting the picture, more or less. Even a few minutes of gameplay. Until it crashes again. But it is getting way too late and we have to finish our real-life real-time collaboration. After that - I'm on my own.

The difference

Since we've got to the point where XDM starts, you might think it's now simply a way of trial and error, catching bugs with GDB. But it's not that simple. To this date, XDM generally had been developed for the x86 machines, and to get it working on Baikal the code had to be modified pretty wildly. And here's why:

• Windows -> Linux
• Intel 386 -> ARM aarch64
• 32-bit -> 64-bit

The first problem was sorted out long ago, since XDM had native Linux server compatibility for quite a while and all other code had been written as OS-independently as possible. You should have a look at my "platform.h" - it's a masterpiece! (written in blood :D) It doesn't even conflict with VALVe's headers!

But the bitness and CPU architecture is the whole other story. Do you know what bytes, words and double words are? On traditional x86 machines they represent 8, 16 and 32-bit integers respectively. In C, however, it's not that easy. C arithmetic types have some non-obvious ambiguity to them, so, it's usually a good idea to write size-dependent code (like data structures, protocols, base classes, interfaces, etc.) in explicit size manner:

// RGBA 4 Bytes
class Color
{
public:
	union
	{
		uint32 integer;// ABGR
		byte array[4];// I'll bite ya! /(0w0)/
		struct
		{
			byte r;
			byte g;
			byte b;
			byte a;
		};
	};

}

But, in classic old C I could've written that aslong int integer;// ABGRor evenlong integer;// ABGR

yeah, you're right: it's better with the "unsigned" part:unsigned long int integer;// ABGR

Now, according to the specification, what will happen on a 64-bit platform? long will become a 64-bit type! (It's history now - who and how designed such unobvious type system, but that's out of this article's scope anyway). So, our Color class becomes 64-bit in size too!

sizeof(Color) == ? // depends.

As long as you're using it within your algorithms, it is probably fine. But wait until you find it in your network code! And this is exactly what happened in my case. The READ_LONG() function was reading too much from a received buffer, corrupting data and throwing exceptions on the way.

Frustrated with such simple yet dangerous mistakes, I needed to make sure there won't be any more of those hiding throughout my huge project. But how do I achieve that?

void WTFCALL GiveFnptrsToDll(enginefuncs_t *pengfuncsFromEngine, globalvars_t *pGlobals)
{
ASSERT(sizeof(int) == 4); // like this???

Surely there must be a solution. Immediately I thought of creating static assertions, only to realize that... there aren't any in C. And I want to maintain the C++98 compatibility. What can I do? Turns out, I'm not the only one who wants to use static compile-time assertions, and there's an entire article dedicated to this method. After investigating a lot into the subject, I've implemented the most suitable (in my opinion) variant. This is what I could do after that:

STATIC_ASSERT(sizeof(short) == 2,	"COMPILE ERROR: this platform has unexpected size of type!");
STATIC_ASSERT(sizeof(int) == 4,		"COMPILE ERROR: this platform has unexpected size of type!");
// this helped A LOT STATIC_ASSERT(sizeof(long) == 4,	"COMPILE ERROR: this platform has unexpected size of type!");
...
#if defined(PTRSIZE_64BIT)
STATIC_ASSERT(sizeof(size_t) == 8,	"COMPILE ERROR: this platform has unexpected size of type!");
#else
STATIC_ASSERT(sizeof(size_t) == 4,	"COMPILE ERROR: this platform has unexpected size of type!");
#endif

Morale of this story: never use non-fixed-size types for network, data, serialization, ... or anything at all. Don't use them! Ever!

// string printing formats are also platform-dependent
#if defined(_WIN32)
#define SIZE_T_S			"%Iu"
#else
#define SIZE_T_S			"%zu"
#endif

So, after ensuring all type-related problems like structure sizes and printf() formats are taken care of, it's time to... get rid of APIs!

U Got To Let The Music

Next problem was music. There is no playback functionality and I WILL NOT SHIP XDM WITHOUT SOUNDTRACKS WE SO CAREFULLY CHOSE!

It's probably well forgotten by now, but Half-Life didn't play mp3s. Its soundtracks were burned onto the game CD (on a CDA partition). It was a somewhat common practice back then (if game music was not in tracker or midi format). Now we take MP3 for granted, but Half-Life started supporting MP3s no earlier than VALVe introduced Steam. Probably in version 1110 or later. And mp3 support came in a very limited way. You could only "play" standard CD tracks. So, in order to address this unfairness, modders had to make something on their own.

A long time ago on a forum far-far away someone posted a proposition to use the FMOD library for playing music. By that time I was considering using Xaudio, but it was way too complicated and I quickly dropped it in favor of FMOD, which has nice, simple enough C-style API and promised future portability. At that time I wasn't even considering running Linux on desktop, let alone porting the game, but still kept writing code that would be easily portable, just in case.

And now this case happened. Once again. Previous case was plain x86 Linux support, and FMOD did have that. Some say it even worked. But this time it's ARM and aarch64. It came as no surprise that FMOD didn't expect it back in 2005 (that's when the version I'm using was released, and Firelight Multimedia dropped FMOD shortly after, AFAIK). ARM wasn't a common thing, and there was no Android.

So, something must be done about it and done quick. I had two options:

• do it properly, write an OOP wrapper that chooses relevant API on demand
• do it quickly, keep current code but wrap every call into IFs with similar purpose

Since the deadline was tomorrow afternoon, the choice was pretty obvious.

Here I have to note: this ARM64 Baikal port I am writing is meant for Xash3D only, as there are no known plans of VALVe adopting ARM CPUs in GoldSource. First thing I actually wanted is a Xash3D API for an MP3 player. Unfortunately, we quickly had to abandon that - not only did we not have eoungh manpower, but hastily designed API is never a good API. Still, we found a workaround, even two of them:

• Use the new client DLL API, gEngfuncs.pfnPrimeMusicStream()
• Directly call console commands: mp3 play/loop

New CL API, as the name suggests, is new, and I'd lose WON Half-Life compatibility if I relied upon it. Unlike the server side, there's no way for a client.dll to know which engine version is loading it, so it'd crash requesting non-existent function pointers immediately. BUT! Since there never were any WON Half-Life releases for Linux, there was no old client API! Not an elegant way of writing things still. Also, Xash3D provides this new, extended API, so... What should I choose?

That's right: I chose both.

	if (mp3player->value == 1.0f)
	{
		gEngfuncs.pfnPrimeMusicStream(path, loop);// XDM3042: this will be overridden for older APIs in XDM
		return true;
	}

- Wait, did you just say you wanted old API compatibility?

Yes, I did. :3

void STUB_pfnPrimeMusicStream(char *szFilename, int looping)// XDM3042
{
	char szCmd[256];
	if (looping > 0)
		_snprintf(szCmd, 256-1, "mp3 loop \"%s\"\n", szFilename);// should be "loopfile" but we might as well allow using original HL soundtracks
	else
		_snprintf(szCmd, 256-1, "mp3 play \"%s\"\n", szFilename);// same with "playfile"
	szCmd[256-1] = 0;
	CLIENT_COMMAND(szCmd);
}

Following my usual principle I just wrote stubs for engine functions missing in the older API. :) Most of them do some actual work. So, when the DLL is compiled with CLDLL_NEWFUNCTIONS it overwrites my stub pointers with real engine functions, when it's not - it doesn't. And it doesn't have to.

There, of course, won't be any seeking, time display or even track changing, because there are obviously no callbacks, but at least we have Play and Stop buttons working.

The hurry

There's actually more to this story. The PC I was borrowing from a friend was scheduled for shipment to a well-known youtuber. :) It all happened very quickly and was arranged while I was on my way to the city, and I'm not complaining - I don't own it and that's okay. But the implications were that I am to work quickly as I don't want to be the person who's wrecking other people's plans.

But the marathon-paced work was not only insanely intense, it was also lots of fun! And who wouldn't want to get his hands on such an oddity! Not to mention to be the first.

Here you can see a bit how this computer looks like. Nothing very special, a normal horizontal ATX PC with 2.5", 3.5" and 5.25" bays. Neat, good for its purpose.

So, to summarize what we have accomplished so far:

• Set up a Baikal computer and installed ALT Linux onto it
• Brought Xash3D to a working condition on aarch64 Baikal
• Got still unfinished VGUI replacement library to work!
• Caught TONS of potential errors using GCC
• Actualized Linux port of XDM
• Found out which parts of Xash3D needs fixing
• Wrote portable Makefiles for all parts XDM (and meanwhile fixed VS projects)
• Met with cool people :)
• Had suffered severe sleep deprivation
• (did a lot more, but I'm too asleep to remember)

Neither a1batross, not mittorn expected OpenVGUI to actually work. :) Good thing XDM uses minimum of its features and does most heavy lifting by itself. I was afraid we'd have trouble with input since I've never tested SDL support, but that didn't bother us at all and I even forgot it was there.

We laughed SO hard when we actually heard bots swearing at us :D (Did you know? (^_^)/ XDM has some kind of soundboards support!)

The last day I spent with this machine one-to-one alone. There were many things to sort out: network, data size inconsistencies, music, etc.

Xash gave me hard time, as usual. :) It still woudn't work with settings.scr properly, not follow HL network protocol strictly, causing all sorts of problems. The Stupid Quake But(tm) popped up all of a sudden again. And it overwrote all of my production screenshots, leaving me with one copy per map.

And finally, even the music started playing!

I was planning, very naively, to finish this work until noon, but ended up working extremely hard until the very night!

While XDM was being "tested" by bots, I was frantically designing CD cover (which, to my utmost disappointment, failed to burn in the end due to poor quality of blank discs!), assembling installation package, compiling release versions (those require quite some tricks), producing materials for publishing (lots of them!), completing new version of the Player Model Pack to ship along. Not to mention I was hoping to finish replacement model for the flamethrower. Time went SO FAST. There's just too much work for single person to handle!

Long past sunset, I burned last files to the disc and rushed outside to make it to the last train...

Outro

Now, as things are (mostly) done, we can proudly state that we did it, and we did it first! Half-Life Xash3D-FWGS and XDM are ported to the Baikal! and I really need some rest /(x_x)\

Who could've thought you'd still be alive this rare (and only one yet) PC would come into my hands and coincidentally go to a blogger I'm subscribed to? :)

Big thanks go to a1batross and mittorn of the FWGS team for the help, my friend who let me borrow the Baikal machine and soon-to-be-disclosed youtuber for letting us torture it for some time. :)

- Please note that this update is not yet available for downloading as it is still too unstable. -

PS: right after the PC was sent away, I've discovered two critical bugs: the SQB-ed flashlight and missing team info (and hence destroyed scoreboard and part of the gameplay) in the Windows version due to unverified commit which fixes Xash3D version and ruins the original goldsource-compatible one. 🤷

PS/2: I've updated photos thumbnails, but due to moddb caching, old ones (which are too big) are still shown in the article. :(