Playthrough Statistics Collection in the Source Engine tutorial

This is an opt-in system I've included with Estranged since Alpha #3, and it's allowed me to get data about how players play the game. Please note: This tutorial is fairly advanced. I've tried to simplify it, but it's pretty messy. If there are any questions I will try and answer them in the comments, and update the article :)

Why Collect Stats

It's useful to get data about how people play your game, and that's why I wrote the Statistics Collector for Estranged. When you first start the game, you're presented with the following dialogue:

If you select "yes", it enables Estranged to push up statistics about gameplay to my server and store them away in a database, as they happen. It's useful for tracking where people die, their health at certain checkpoints, seeing which route players generally take, and for seeing if they pick up on small easter eggs in levels.

Prerequisites

Things to have ready before we start.

A web server with PHP and MySQL

This is the business end of the system- an HTTP server that we can hit with data, and a database to store it. Most (if not all, it's everywhere) hosts support PHP, and the majority of them give you access to a MySQL instance. I won't go into detail about actually getting a server and who to get one from because there's so much about that on the Internet already.

A Source Engine mod

This is a pretty easy feature to implement into Source, but there are a few prerequisites - one of which is that you have to compile and edit the source code for the engine. That means getting your hands dirty with C++. There's an excellent guide about setting up a mod with Visual Studio here.

libcurl

If you have a mod set up with the source code accessible, you need to add the cURL libraries to the mod. cURL is a cross-platform library allowing you to make requests to remote HTTP servers. There's a very easy to follow guide on the Valve Developer Wiki about adding it to your mod here.

Implementing

We're going to be creating a "Logical Entity" in Source; that's one that you can plop in a map in Hammer, but it doesn't have a model/physics mesh. I highly recommend checking out this article before reading any furthur as it's an excellent read.

The C++ Entity

The full source for the Statistics Collector in Estranged can be found here. I'm not going through the whole code, but I will explain the important parts of it. If you want to know more, leave a comment and I'll get back to you.

To start off with, below the includes there's a line of code saying:

That tells the code where the stats need to be sent.

When the entity is constructed, it tells the Engine that it wants to listen to the player_hurt event. This is to allow us to log player deaths. There is a player_death event, but I had difficulty getting that to work.

That code gets recieved here:

It first checks if we're getting the response from the right event, then checks if the player is really dead. It then calls our SendStatistic() method with the unfortunate news.

This method gets called whenever we want to send a statistic back to the web server. It first checks our stats_enabled CVAR, which needs to be set to 1 for stats to be enabled. This can be hooked up to a VGUI screen that asks the user about stats when the game starts up, and sets it based on their response.

the ThreadExecute stuff executes the cURL HTTP request in a different thread - this is because the request is synchronus, so if we were to fire it off in the current thread the game would pause for a few seconds while it completed. Which would be awful. So this makes it unobtrusive.

The whole cURLThread method basically builds up a collection of data about the player. The weapon, the current time, the map, their position etc, and builds it into a string like below:

It then fires of an HTTP request to log it.

The FGD for Hammer

Once it has been added to your game, you need to add an entry to your FGD file. More about what that is here.

My FGD entry looks like this:

The PHP Code

I'm going to assume you have access to a database already, so here's the table schema to set up the collector:

Once that's in place, you can add the simple PHP script to add the data.

And that's the server-side stuff done.

Using it In-Game

To use the entity in game, simply make one instance of it in a map, and tell other entities (such as triggers) to send the "Event" input to it with a label for the event. The above screenshot shows all of the inputs set up to hit the statistics collector in the first level of Estranged.

Aggregating the Data

Displaying the data is just a case of using either a MySQL administration system, or a PHP script to display the stats. That's how the Estranged statistics page works, but you might not want to show the data once it's collected.

Some example queries you can run on the data:

Number of deaths per map:

The number of unique players:

... there are lots of things you can do. You can search for specific events, and compare them against other events for instance, to compare routes players took.

If there's anyone looking for the VGUI half of this (the bit that asks the user whether they want to send stats, setting the stats_enabled CVAR), I'll write another article documenting it.