Estranged, the story of a lone fisherman, stranded on a mysterious island during a violent storm. Explore rich environments and meet the curious inhabitants of the island as you find a way back to the mainland.

Report article RSS Feed Playthrough Statistics Collection in the Source Engine

This is an opt-in system to get data about how players play your mod.

Posted by AlanEdwardes on Oct 6th, 2012
Advanced Server Side Coding.

This is an opt-in system I've included with Estranged since Alpha #3, and it's allowed me to get data about how players play the game. Please note: This tutorial is fairly advanced. I've tried to simplify it, but it's pretty messy. If there are any questions I will try and answer them in the comments, and update the article :)

Why Collect Stats


It's useful to get data about how people play your game, and that's why I wrote the Statistics Collector for Estranged. When you first start the game, you're presented with the following dialogue:

Estranged Statistics Dialouge

If you select "yes", it enables Estranged to push up statistics about gameplay to my server and store them away in a database, as they happen. It's useful for tracking where people die, their health at certain checkpoints, seeing which route players generally take, and for seeing if they pick up on small easter eggs in levels.

Prerequisites


Things to have ready before we start.

A web server with PHP and MySQL


This is the business end of the system- an HTTP server that we can hit with data, and a database to store it. Most (if not all, it's everywhere) hosts support PHP, and the majority of them give you access to a MySQL instance. I won't go into detail about actually getting a server and who to get one from because there's so much about that on the Internet already.

A Source Engine mod


This is a pretty easy feature to implement into Source, but there are a few prerequisites - one of which is that you have to compile and edit the source code for the engine. That means getting your hands dirty with C++. There's an excellent guide about setting up a mod with Visual Studio here.

libcurl


If you have a mod set up with the source code accessible, you need to add the cURL libraries to the mod. cURL is a cross-platform library allowing you to make requests to remote HTTP servers. There's a very easy to follow guide on the Valve Developer Wiki about adding it to your mod here.

Implementing


We're going to be creating a "Logical Entity" in Source; that's one that you can plop in a map in Hammer, but it doesn't have a model/physics mesh. I highly recommend checking out this article before reading any furthur as it's an excellent read.

The C++ Entity


The full source for the Statistics Collector in Estranged can be found here. I'm not going through the whole code, but I will explain the important parts of it. If you want to know more, leave a comment and I'll get back to you.

To start off with, below the includes there's a line of code saying:

cpp code:
#define ENDPOINT "http://example.com/collector.php?collect"

That tells the code where the stats need to be sent.

When the entity is constructed, it tells the Engine that it wants to listen to the player_hurt event. This is to allow us to log player deaths. There is a player_death event, but I had difficulty getting that to work.

cpp code:
CStatisticsCollector()
{
  ListenForGameEvent("player_hurt");
}
 

That code gets recieved here:

cpp code:
void CStatisticsCollector::FireGameEvent( IGameEvent* e )
{
  CBasePlayer *player = UTIL_GetLocalPlayer();
  if ( Q_strcmp( "player_hurt", e->GetName() ) && player->GetHealth() == 0 )
  {
    // The player has unfortunately passed away
    this->SendStatistic("player_death");
  }
}
 

It first checks if we're getting the response from the right event, then checks if the player is really dead. It then calls our SendStatistic() method with the unfortunate news.

cpp code:
void CStatisticsCollector::SendStatistic( const char *m_event )
{
  if(stats_enabled.GetInt() > 0)
  {
    DevMsg("Statistics Collectior: cURL thread queued.\n");
    ThreadExecute(this, &CStatisticsCollector::cURLThread, m_event);
  }
  else
  {
    DevMsg("Statistics Collectior: Events disabled, ignoring.\n");
  }
}
 

This method gets called whenever we want to send a statistic back to the web server. It first checks our stats_enabled CVAR, which needs to be set to 1 for stats to be enabled. This can be hooked up to a VGUI screen that asks the user about stats when the game starts up, and sets it based on their response.

the ThreadExecute stuff executes the cURL HTTP request in a different thread - this is because the request is synchronus, so if we were to fire it off in the current thread the game would pause for a few seconds while it completed. Which would be awful. So this makes it unobtrusive.

The whole cURLThread method basically builds up a collection of data about the player. The weapon, the current time, the map, their position etc, and builds it into a string like below:

code:
Example.com Oct 06 20:05:27 2012&m=sp01thebeginning

It then fires of an HTTP request to log it.

The FGD for Hammer


Once it has been added to your game, you need to add an entry to your FGD file. More about what that is here.

My FGD entry looks like this:

code:
@PointClass base(Targetname) iconsprite("editor/multisource.vmt") = statistics_collector : "Statistics Collector"
[
  input Event(string) : "Send an event"
]

The PHP Code


I'm going to assume you have access to a database already, so here's the table schema to set up the collector:

sql code:
CREATE TABLE IF NOT EXISTS `statistics_collector` (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `steam_id` varchar(255) NOT NULL,
  `ip` varchar(255) NOT NULL,
  `event` varchar(255) NOT NULL,
  `position` varchar(255) NOT NULL,
  `health` int(3) NOT NULL,
  `time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
  `local_ds` varchar(255) NOT NULL,
  `map` varchar(255) NOT NULL,
  `weapon` varchar(255) NOT NULL,
  PRIMARY KEY (`id`)
)
 

Once that's in place, you can add the simple PHP script to add the data.

php code:
<?php
$mysqli = new mysqli('database-server', 'database-username', 'database-password', 'database-name');
header('Content-type: text/plain');

$mapping = array(
  'e' => 'event',
  's' => 'steam_id',
  'p' => 'position',
  'h' => 'health',
  'd' => 'local_ds',
  'm' => 'map',
  'w' => 'weapon'
);

$fields = array('ip');
$values = array('"' . $_SERVER['REMOTE_ADDR'] . '"');
foreach($mapping as $i => $field)
{
  if(isset($_GET[$i]))
  {
    $fields[] = $field;
    $values[] = '"' . $mysqli->real_escape_string($_GET[$i]) . '"';
  }
}

$mysqli->query('
  INSERT INTO statistics_collector('
. implode($fields, ',') . ')
  VALUES('
. implode($values, ',') . ');
'
);
?>
 

And that's the server-side stuff done.

Using it In-Game


Statistics Collector in hammer

To use the entity in game, simply make one instance of it in a map, and tell other entities (such as triggers) to send the "Event" input to it with a label for the event. The above screenshot shows all of the inputs set up to hit the statistics collector in the first level of Estranged.

Aggregating the Data


Displaying the data is just a case of using either a MySQL administration system, or a PHP script to display the stats. That's how the Estranged statistics page works, but you might not want to show the data once it's collected.

Some example queries you can run on the data:

Number of deaths per map:

sql code:
SELECT COUNT(*) `num`, `map` FROM `statistics_collector` WHERE `event` = 'player_death' GROUP BY `map` ORDER BY `num` DESC

The number of unique players:

sql code:
SELECT COUNT(DISTINCT `steam_id`) `num` FROM `statistics_collector`

... there are lots of things you can do. You can search for specific events, and compare them against other events for instance, to compare routes players took.

If there's anyone looking for the VGUI half of this (the bit that asks the user whether they want to send stats, setting the stats_enabled CVAR), I'll write another article documenting it.

Post comment Comments
flippedoutkyrii
flippedoutkyrii Oct 6 2012, 11:37pm says:

Sounds like a nice concept, and could help modders better create an environment with suitable game-play styles :)

+2 votes     reply to comment
ninjaman999
ninjaman999 Oct 7 2012, 3:39am says:

Brilliant. This would help iron out difficulty spikes and points of interest for map design and player progression! :D

+4 votes     reply to comment
Jack5500
Jack5500 Oct 7 2012, 8:58am says:

Nice idea. Maybe it would be nice, if you could provide a template like set-up'ed project which is fixed for visual studio

+1 vote     reply to comment
Post a Comment
click to sign in

You are not logged in, your comment will be anonymous unless you join the community today (totally free - or sign in with your social account on the right) which we encourage all contributors to do.

2000 characters limit; HTML formatting and smileys are not supported - text only

Icon
Half-Life 2 Icon
Platforms
Windows, Mac, Linux
Developed By
AlanEdwardes
Contact
Send Message
Official Page
Iamestranged.com
Release Date
Released Jan 15, 2014
Mod Watch
Track this mod
Tutorial
Browse
Tutorials
Report Abuse
Report article
Related Mods
Estranged (Half-Life 2)
Estranged Half-Life 2 - Single Player Adventure
Related Games
Half-Life 2
Half-Life 2 Single & Multiplayer First Person Shooter