The server got overloaded in two ways;
1) Way, waaay too much traffic and too many connections
2) Way, waaay too much cpu usage.
For the traffic, I did this:
I moved all the game binaries out to an Amazon S3 cloud thingie. This scales easily with out needs, and even has support for a CDN in Amazon CloudFront. A CDN (Content Delivery Network) is basically a bunch of servers all over the world which redirect so you download the files from the server nearest to you.
I haven’t been able to move the resources (sounds and music yet) because those used a rather ridiculous auto updating scheme that was different from the rest of the game. I will fix this today.
Minecraft Classic is also offline for the time being since I wasn’t able to get it to download from the S3 easily. I will try to get this up today as well, but I can’t promise anything there. It is coming back, though.
For the CPU load, I did this:
The problem was that I had written my own database early on before I even had decided to sell the game yet, and it kind of stuck around. I sat down and set up a proper RDB database (mysql). Currently, this is also running via Amazon Cloud.
The first version of the DB hit the CPU cap pretty fast until I realized I could scale it up to a much faster database with just a couple of clicks. Now it’s currently configured to be quite a bit faster than we actually need, but since I like the idea of actually having MORE power than needed somewhere, I’m keeping it that way for a few days.
After I temporarily took out the sound and music updating, the website got a lot faster, but before that payments were queuing up, so it could take quite some time for some people to actually get their game after they bought it.
What happened is that paypay tried to send the notification immediately, but when the system is busy, it adds the notification to an internal queue in paypal’s system, and resends it after a short period of time. If it fails a SECOND time, it increases the counter and tries even later. So if you got unlucky, it could time out three times in a row, and you’d have to wait a fairly long time. All the payments are still secure, but they just didn’t get processed immediately.
If you bought the game and didn’t receive it immediately, I’m sorry to keep you waiting. Please wait until tomorrow or so before contacting us, though.. We’re getting a LOT of emails about accounts that have already been updated automatically when we check them.
TkTech is pretty much done with his rewrite which will give minecraft.net a much more modern codebase. We’ll set that version up on a separate server and test it, then cut over when it works. We might need help testing it, and then I thought I might ask you guys to help out, if you don’t mind. :-D