Themes
Report article RSS Feed Platform Update

News about backend changes that we are implementing to make ModDB, Indiedb & Desura faster for you to use!

Posted by Greg on Mar 16th, 2011

A couple of big changes are on the way with our platform as we strive to keep pace with the growth of our community and improve the quality of the service that we deliver.

Some of you may have noticed that there have been a few issues with site slowdowns during the past month (with Desura and IndieDB both growing), and we have isolated this problem some time ago - however the cause and subsequent fix has been time consuming to implement.

Bring on nginx! What is nginx? Previously, all of the DesuraNET sites used Apache to serve our dynamic content, with a combination of lighttpd and squid serving out media (images/videos/downloads). Apache (though amazing) is a rather large piece of software which can quickly become a ram hog, limiting the number of maxclients you can have connected at the same time. Because of this we've had to look elsewhere, and after much experimentation we settled on nginx, a lightweight Apache made by a lone Russian hacker.


Nginx is a webserver that uses a fastcgi wrapper to spawn PHP - upto twice as fast as Apache, and during busy periods, easily 10 times as fast as Apache. This is great, as it means we can reduce our webserving hardware requirement, while keeping spare capacity for peak times, allowing for resources to be shifted elsewhere.

One of these resource shifts is into an Anycast CDN. What does this mean? Well it means that if you are in Europe, when you visit a DesuraNET site or use the Desura client, all the images/video you see will come down via the EU - providing you with much faster image rendering and load speeds. We can add additional locations in the future to further capacity and reduce media serving time. This CDN should be operational within a the next 1 ~ 2 months.

As always if you notice anything odd like images not loading, http errors, uploads failing please let us know as this is a MAJOR backend change! Thanks.

Post comment Comments
ForK
ForK Mar 16 2011, 7:52am says:

ahh technology! .. Greg bless the intrawebs...

+3 votes     reply to comment
INtense!
INtense! Mar 16 2011, 7:56am says:

Previously our servers started to struggle around 5000 connections and Apache was constantly playing with fire. Instantly after this change server load has dropped from 2 to 0.2 which is incredible.

Speed and our ability to scale should be significantly enhanced if all goes well.

+5 votes     reply to comment
mgs2
mgs2 Mar 16 2011, 9:09am says:

I don't know if it is only me but I can't see images from different pages here in moddb.

+2 votes     reply to comment
INtense!
INtense! Mar 16 2011, 9:25am replied:

can you email a screenshot to support@moddb.com

Images should not be affected as they run on a separate server.

+3 votes     reply to comment
mgs2
mgs2 Mar 18 2011, 7:33am replied:

Well, the next day after posting my previous comment pictures show up normally again. Hope that Moddb will continue to grow and change the way of Video Games!

+1 vote     reply to comment
Cameron:D
Cameron:D Mar 16 2011, 9:31am says:

Ahhh. I love nginx. Its stability and resource usage under high loads is amazing.

+3 votes     reply to comment
Metalspy
Metalspy Mar 16 2011, 7:57pm says:

"made by a lone Russian hacker."
I thought hackers were people challenging digital security? :P

Anyway it sounds like a great improvement if the server load has dropped from 2 to 0.2 (whatever those numbers exactly mean) ;) So yay for the wonders of ever-improving technology.

+3 votes     reply to comment
Katana_
Katana_ Mar 18 2011, 4:25pm replied:

Read the jargon file, young blood. ;)

Catb.org

+1 vote     reply to comment
Heaney
Heaney Mar 16 2011, 8:04pm says:

Considering I posted an entire article yesterday and it just gave me some 505 error and I lost the entire thing- I do NOT like this...

Or was it even implemented yesterday? Was that just a random error?

+1 vote     reply to comment
Greg
Greg Mar 16 2011, 8:42pm replied:

I apologise for the error you received Heaney - there were some glitches that appeared about an hour ago which have been corrected now. If you get an error after submitting an article you can always try to hit reload to re-submit the data which should work.

+3 votes     reply to comment
Salsa_Shark
Salsa_Shark Mar 16 2011, 11:23pm says:

In soviet Russia nginx serves you!

+8 votes     reply to comment
AirborneSn1p3r
AirborneSn1p3r Mar 17 2011, 7:19am says:

great to see this

+1 vote     reply to comment
Relto
Relto Mar 17 2011, 6:23pm says:

SCIENCE

+2 votes     reply to comment
SCΛRECROW
SCΛRECROW Mar 18 2011, 4:39am says:

More speed. Excellent.

+1 vote     reply to comment
Kissaki
Kissaki Mar 20 2011, 6:25am says:

“after much experimentation”
Did you also try/test cherokee?
I’d be interested in how it performed in comparison to nginx.

+1 vote     reply to comment
Greg
Greg Mar 20 2011, 11:59am replied:

We tried Cherokee and Nginx. We spent about 2 weeks debugging odd XML/SSL issues with Cherokee, aswell as a number of other issues.

In the end, we would have preferred to use Cherokee over nginx due to its ease of configuration, supposed better performance, soft restart abilities and nifty admin UI, however the bugs we encountered with it forced us to go with nginx - which to it's credit, had 0 bugs, and was quite simple to deploy!

+2 votes     reply to comment
cobrafang1961
cobrafang1961 Feb 1 2012, 6:41am says:

I have PB heartbeats stopping, help!!!

+1 vote     reply to comment
Post a Comment
click to sign in

You are not logged in, your comment will be anonymous unless you join the community today (totally free - or sign in with your social account on the right) which we encourage all contributors to do.

2000 characters limit; HTML formatting and smileys are not supported - text only

News
Browse
News
Report Abuse
Report article
Related Groups
Desura
Desura Official group with 10,083 members
IndieDB
IndieDB Official group with 1,997 members
ModDB
ModDB Official group with 3,801 members