Post tutorial RSS Playthrough Statistics Collection in the Source Engine

This is an opt-in system to get data about how players play your mod.

Posted by on - Advanced Server Side Coding

This is an opt-in system I've included with Estranged since Alpha #3, and it's allowed me to get data about how players play the game. Please note: This tutorial is fairly advanced. I've tried to simplify it, but it's pretty messy. If there are any questions I will try and answer them in the comments, and update the article :)

Why Collect Stats

It's useful to get data about how people play your game, and that's why I wrote the Statistics Collector for Estranged. When you first start the game, you're presented with the following dialogue:

Estranged Statistics Dialouge

If you select "yes", it enables Estranged to push up statistics about gameplay to my server and store them away in a database, as they happen. It's useful for tracking where people die, their health at certain checkpoints, seeing which route players generally take, and for seeing if they pick up on small easter eggs in levels.


Things to have ready before we start.

A web server with PHP and MySQL

This is the business end of the system- an HTTP server that we can hit with data, and a database to store it. Most (if not all, it's everywhere) hosts support PHP, and the majority of them give you access to a MySQL instance. I won't go into detail about actually getting a server and who to get one from because there's so much about that on the Internet already.

A Source Engine mod

This is a pretty easy feature to implement into Source, but there are a few prerequisites - one of which is that you have to compile and edit the source code for the engine. That means getting your hands dirty with C++. There's an excellent guide about setting up a mod with Visual Studio here.


If you have a mod set up with the source code accessible, you need to add the cURL libraries to the mod. cURL is a cross-platform library allowing you to make requests to remote HTTP servers. There's a very easy to follow guide on the Valve Developer Wiki about adding it to your mod here.


We're going to be creating a "Logical Entity" in Source; that's one that you can plop in a map in Hammer, but it doesn't have a model/physics mesh. I highly recommend checking out this article before reading any furthur as it's an excellent read.

The C++ Entity

The full source for the Statistics Collector in Estranged can be found here. I'm not going through the whole code, but I will explain the important parts of it. If you want to know more, leave a comment and I'll get back to you.

To start off with, below the includes there's a line of code saying:

cpp code:
#define ENDPOINT ""

That tells the code where the stats need to be sent.

When the entity is constructed, it tells the Engine that it wants to listen to the player_hurt event. This is to allow us to log player deaths. There is a player_death event, but I had difficulty getting that to work.

cpp code:

That code gets recieved here:

cpp code:
void CStatisticsCollector::FireGameEvent( IGameEvent* e )
  CBasePlayer *player = UTIL_GetLocalPlayer();
  if ( Q_strcmp( "player_hurt", e->GetName() ) && player->GetHealth() == 0 )
    // The player has unfortunately passed away

It first checks if we're getting the response from the right event, then checks if the player is really dead. It then calls our SendStatistic() method with the unfortunate news.

cpp code:
void CStatisticsCollector::SendStatistic( const char *m_event )
  if(stats_enabled.GetInt() > 0)
    DevMsg("Statistics Collectior: cURL thread queued.\n");
    ThreadExecute(this, &CStatisticsCollector::cURLThread, m_event);
    DevMsg("Statistics Collectior: Events disabled, ignoring.\n");

This method gets called whenever we want to send a statistic back to the web server. It first checks our stats_enabled CVAR, which needs to be set to 1 for stats to be enabled. This can be hooked up to a VGUI screen that asks the user about stats when the game starts up, and sets it based on their response.

the ThreadExecute stuff executes the cURL HTTP request in a different thread - this is because the request is synchronus, so if we were to fire it off in the current thread the game would pause for a few seconds while it completed. Which would be awful. So this makes it unobtrusive.

The whole cURLThread method basically builds up a collection of data about the player. The weapon, the current time, the map, their position etc, and builds it into a string like below:

code: Oct 06 20:05:27 2012&m=sp01thebeginning

It then fires of an HTTP request to log it.

The FGD for Hammer

Once it has been added to your game, you need to add an entry to your FGD file. More about what that is here.

My FGD entry looks like this:

@PointClass base(Targetname) iconsprite("editor/multisource.vmt") = statistics_collector : "Statistics Collector"
  input Event(string) : "Send an event"

The PHP Code

I'm going to assume you have access to a database already, so here's the table schema to set up the collector:

sql code:
CREATE TABLE IF NOT EXISTS `statistics_collector` (
  `steam_id` varchar(255) NOT NULL,
  `ip` varchar(255) NOT NULL,
  `event` varchar(255) NOT NULL,
  `position` varchar(255) NOT NULL,
  `health` int(3) NOT NULL,
  `local_ds` varchar(255) NOT NULL,
  `map` varchar(255) NOT NULL,
  `weapon` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)

Once that's in place, you can add the simple PHP script to add the data.

php code:
$mysqli = new mysqli('database-server', 'database-username', 'database-password', 'database-name');
header('Content-type: text/plain');

$mapping = array(
  'e' => 'event',
  's' => 'steam_id',
  'p' => 'position',
  'h' => 'health',
  'd' => 'local_ds',
  'm' => 'map',
  'w' => 'weapon'

$fields = array('ip');
$values = array('"' . $_SERVER['REMOTE_ADDR'] . '"');
foreach($mapping as $i => $field)
    $fields[] = $field;
    $values[] = '"' . $mysqli->real_escape_string($_GET[$i]) . '"';

  INSERT INTO statistics_collector('
. implode($fields, ',') . ')
. implode($values, ',') . ');

And that's the server-side stuff done.

Using it In-Game

Statistics Collector in hammer

To use the entity in game, simply make one instance of it in a map, and tell other entities (such as triggers) to send the "Event" input to it with a label for the event. The above screenshot shows all of the inputs set up to hit the statistics collector in the first level of Estranged.

Aggregating the Data

Displaying the data is just a case of using either a MySQL administration system, or a PHP script to display the stats. That's how the Estranged statistics page works, but you might not want to show the data once it's collected.

Some example queries you can run on the data:

Number of deaths per map:

sql code:
SELECT COUNT(*) `num`, `map` FROM `statistics_collector` WHERE `event` = 'player_death' GROUP BY `map` ORDER BY `num` DESC

The number of unique players:

sql code:
SELECT COUNT(DISTINCT `steam_id`) `num` FROM `statistics_collector`

... there are lots of things you can do. You can search for specific events, and compare them against other events for instance, to compare routes players took.

If there's anyone looking for the VGUI half of this (the bit that asks the user whether they want to send stats, setting the stats_enabled CVAR), I'll write another article documenting it.


Sounds like a nice concept, and could help modders better create an environment with suitable game-play styles :)

Reply Good karma Bad karma+2 votes

Brilliant. This would help iron out difficulty spikes and points of interest for map design and player progression! :D

Reply Good karma Bad karma+4 votes

Nice idea. Maybe it would be nice, if you could provide a template like set-up'ed project which is fixed for visual studio

Reply Good karma Bad karma+1 vote
Post a comment
Sign in or join with:

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.