Post news RSS Releasing code in a live environment

3 years, 853 million page views, 815 releases, 1 day downtime (for patching)

Posted by on

Running a massively multiplayer game is difficult. You need good up-time, but also need to continuously update the game and add new content.

The server is busy. We've had 853 million page views in our three years since launch and we currently average 800 page views a minute with over 1.2 million a day - and that's only active pages being viewed; not including game notifications, passive chat, or any ingame popups.

We've just done our 815th release to live; I imagine even our players don't know we release something on average every 1.4 days. Some are minor bug fixes, others full UI refreshes, of which we have done 3:

Illyriad on a Kindle TouchWe are available on many platforms, with an HTML5 website game, a Chrome Webstore App, Firefox App, a Facebook game, and Windows 8 App; all of which run on the same single sharded world, and all of which need to be kept up to date and in sync.

We have to ensure with each change everything keeps working on all the different platforms, different browsers and different devices - Illyriad even works on a Kindle Touch!

We run a full Microsoft stack, have changed CDN 4 times, and during these three years we've had 1 day of cumulative downtime, mostly for security patches, although our longest period was for completely moving hosting providers to Hivelocity for our dedicated servers and Windows Azure for our cloud based needs and CDN - lots of live data to move.

All through this the players keep playing.

Blissfully unaware of all the changes going on. Sending their troops on 58 million combat missions, their traders on 41 million missions and building their 33 million buildings, all the while chatting 213 million words together.

Each update is transparent and doesn't interrupt the players' play time for patching. People live busy lives, and they have chosen to give their precious time to play the game and that must be respected.

They don't want to spend that time waiting for downloads, patching and updates.

They just want to play.

Maintaining a uninterrupted service, with huge concurrency and transparent updates is hard, but its worth it!

Post a comment
Sign in or join with:

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.