News about backend changes that we are implementing to make ModDB, Indiedb & Desura faster for you to use!

Posted by on

A couple of big changes are on the way with our platform as we strive to keep pace with the growth of our community and improve the quality of the service that we deliver.

Some of you may have noticed that there have been a few issues with site slowdowns during the past month (with Desura and IndieDB both growing), and we have isolated this problem some time ago - however the cause and subsequent fix has been time consuming to implement.

Bring on nginx! What is nginx? Previously, all of the DesuraNET sites used Apache to serve our dynamic content, with a combination of lighttpd and squid serving out media (images/videos/downloads). Apache (though amazing) is a rather large piece of software which can quickly become a ram hog, limiting the number of maxclients you can have connected at the same time. Because of this we've had to look elsewhere, and after much experimentation we settled on nginx, a lightweight Apache made by a lone Russian hacker.


Nginx is a webserver that uses a fastcgi wrapper to spawn PHP - upto twice as fast as Apache, and during busy periods, easily 10 times as fast as Apache. This is great, as it means we can reduce our webserving hardware requirement, while keeping spare capacity for peak times, allowing for resources to be shifted elsewhere.

One of these resource shifts is into an Anycast CDN. What does this mean? Well it means that if you are in Europe, when you visit a DesuraNET site or use the Desura client, all the images/video you see will come down via the EU - providing you with much faster image rendering and load speeds. We can add additional locations in the future to further capacity and reduce media serving time. This CDN should be operational within a the next 1 ~ 2 months.

As always if you notice anything odd like images not loading, http errors, uploads failing please let us know as this is a MAJOR backend change! Thanks.

Comments
ForK
ForK

ahh technology! .. Greg bless the intrawebs...

Reply Good karma Bad karma+3 votes
INtense! Staff
INtense!

Previously our servers started to struggle around 5000 connections and Apache was constantly playing with fire. Instantly after this change server load has dropped from 2 to 0.2 which is incredible.

Speed and our ability to scale should be significantly enhanced if all goes well.

Reply Good karma+5 votes
mgs2
mgs2

I don't know if it is only me but I can't see images from different pages here in moddb.

Reply Good karma Bad karma+2 votes
INtense! Staff
INtense!

can you email a screenshot to support@moddb.com

Images should not be affected as they run on a separate server.

Reply Good karma+3 votes
mgs2
mgs2

Well, the next day after posting my previous comment pictures show up normally again. Hope that Moddb will continue to grow and change the way of Video Games!

Reply Good karma Bad karma+1 vote
Cameron:D
Cameron:D

Ahhh. I love nginx. Its stability and resource usage under high loads is amazing.

Reply Good karma Bad karma+3 votes
Metalspy
Metalspy

"made by a lone Russian hacker."
I thought hackers were people challenging digital security? :P

Anyway it sounds like a great improvement if the server load has dropped from 2 to 0.2 (whatever those numbers exactly mean) ;) So yay for the wonders of ever-improving technology.

Reply Good karma Bad karma+3 votes
Katana_
Katana_

Read the jargon file, young blood. ;)

Catb.org

Reply Good karma Bad karma+1 vote
Heaney

Considering I posted an entire article yesterday and it just gave me some 505 error and I lost the entire thing- I do NOT like this...

Or was it even implemented yesterday? Was that just a random error?

Reply Good karma Bad karma+1 vote
Greg Author
Greg

I apologise for the error you received Heaney - there were some glitches that appeared about an hour ago which have been corrected now. If you get an error after submitting an article you can always try to hit reload to re-submit the data which should work.

Reply Good karma+3 votes
Salsa_Shark
Salsa_Shark

In soviet Russia nginx serves you!

Reply Good karma Bad karma+8 votes
AirborneSn1p3r
AirborneSn1p3r

great to see this

Reply Good karma Bad karma+1 vote
Relto
Relto

SCIENCE

Reply Good karma Bad karma+2 votes
SCΛRECROW
SCΛRECROW

More speed. Excellent.

Reply Good karma Bad karma+1 vote
Kissaki
Kissaki

“after much experimentation”
Did you also try/test cherokee?
I’d be interested in how it performed in comparison to nginx.

Reply Good karma Bad karma+1 vote
Greg Author
Greg

We tried Cherokee and Nginx. We spent about 2 weeks debugging odd XML/SSL issues with Cherokee, aswell as a number of other issues.

In the end, we would have preferred to use Cherokee over nginx due to its ease of configuration, supposed better performance, soft restart abilities and nifty admin UI, however the bugs we encountered with it forced us to go with nginx - which to it's credit, had 0 bugs, and was quite simple to deploy!

Reply Good karma+2 votes
cobrafang1961
cobrafang1961

I have PB heartbeats stopping, help!!!

Reply Good karma Bad karma+1 vote
Post a comment
Sign in or join with:

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.

News
Browse
News
New
Post news
Share
Related Groups
Desura
Desura Entertainment & Press with 10,149 members
Indie DB
Indie DB Official with 2,375 members
Mod DB
Mod DB Official with 4,146 members