Post news RSS Technical Update #1

First of many technical updates with inside info on how ModDB's systems work, and responses to some of the biggest feedback requests on the site!

Posted by on

So i'm Greg, the sys admin working on ModDB servers and software. It's a job i've been doing behind the scenes for 8 years now (99.8% uptime last year), and I figured a few might be interested to know what is going on. Hence my post (more will come on a semi-regular basis) if you find this interesting and want me to share more insight into what it takes to run a site the size of ModDB.

Over the last couple of months we have:

  • Virtualized and made redundant all web / batch / media and streaming servers so we can add and change our setup with no downtime.
  • Cached and spread the load of delivering millions of images across many servers, instead of relying on one EPIC master to do the grunt work. To give you an indication of the impact this has, check out the image below which shows the difference adding 1 webserver made:

We are also shortly about to undertake ModDB's biggest upgrade yet which will:

  • Replace our primary database server so it goes from 16 to 24 cores. We are also considering upgrading MySQL 5 to 5.5 which will allow us to utilize all 24 cores (at the moment we only utilize 4!!)
  • Replace our monster backend storage system with a SAN that will provide a 5 times increase in storage capacity for everything from files, to videos and more.

This will mean:

  • A faster site.
  • We can consider upping the image size limits.
  • Eventually a better video serving solution.

These initiatives only scratch the surface with regards to what we are going to do over the next few months (i.e. fix dodgy ads, update videos, more download servers), so track this blog for updates.

Comments
Darkbladecr
Darkbladecr

great stuff, interesting post, thanks for the update ;)

Reply Good karma Bad karma+3 votes
R3ven
R3ven

Yes, we will be able to upload larger images! :D

This is great, meaning mods and indie games can post higher resolution, more detailed screenshots and renders :D

And the site will be becoming faster, which is great to hear, as the site is already pretty fast in my opinion.

Reply Good karma Bad karma+3 votes
KamiSLO
KamiSLO

Your are serving videos from your own servers?

Any particular reason your are not using a CDN (Akamai / CloudFront or something similar)?

Reply Good karma Bad karma+2 votes
Greg Author
Greg

We currently use a CDN operated by Akamai, however it is "sponsored" by our Advertising partner.

We used to operate our own Geo-CDN out of Australia, the US & EU - which we are looking at re-visiting due to the quality of the current video CDN's advertising not being what we are happy with.

Reply Good karma+4 votes
Arxae
Arxae

you soo stole the idea from wolfire :D so it means its awesome ^^

Reply Good karma Bad karma+2 votes
cW#Ravenblood

cool!

Reply Good karma Bad karma+2 votes
Ark_
Ark_

Its nice to get some insights on how things are going behind the scenes.

Reply Good karma Bad karma+3 votes
Noktis
Noktis

yeah ^^

Reply Good karma Bad karma+1 vote
Anddos
Anddos

how much electricity does moddb use?

Reply Good karma Bad karma+2 votes
INtense! Staff
INtense!

No idea, but we run about 15 servers (mostly for bandwidth) so it would be a decent amount.

Reply Good karma+1 vote
justgoaway
justgoaway

Great work!

Reply Good karma Bad karma+1 vote
Warrozo
Warrozo

Great work guys, look forward to working with you!

Reply Good karma Bad karma+1 vote
oWn4g3
oWn4g3

I never had performance issues, but improvements are always welcome :)

Reply Good karma Bad karma+2 votes
INtense! Staff
INtense!

we haven't had performance issues, site should always be zippy (0.05 render times)... but it never hurts to go even faster!

Reply Good karma+2 votes
BigBird
BigBird

High Five Greg :D

Always been working hard behind he scenes. When we going for another beer anyways?? Haha

Reply Good karma Bad karma+1 vote
SCΛRECROW
SCΛRECROW

Sounds like a plan. Good work.

Reply Good karma Bad karma+1 vote
rellik_pt
rellik_pt

moddb is spamming now?


"weah22 sent you a message on Mod DB

jeniferweah @ hotmail . fr

Hello dear,

i came across your contact via(moddb.com

)while browsing in the internet
now and it captured my interest and i become motivated to reach you.I'm miss jenifer weah by name,please i will like us to hold a good relationship with a real love,your profile sound so gentle to me that was why i fell very much eager to write you.,contact me through my personal box(jeniferweah @ hotmail . fr) for more introduction,also i will send my pictures to you for this will forster the relationship and also bring us together to know each other better,i look forward for
your response my dear,age or colour even distance can't deny any genuine love,so please lets give our self a trial,thanks till i hear from you

miss jenifer
--------------------

To reply to this message, follow the link below:
Moddb.com

To control the emails you receive from Mod DB, follow the link below:
Moddb.com

Thanks,
The Mod DB Team"

Reply Good karma Bad karma+1 vote
Henley
Henley

No, we don't spam. This would be another member send me a PM about it please.

Reply Good karma Bad karma+1 vote
R3ven
R3ven

Oh god, not that bot again. Same thing happened to me last year sometime.

Almost the same message with minor differences too.

Reply Good karma Bad karma+1 vote
Greg Author
Greg

Thanks for alerting us to this, we'll take a look.

Reply Good karma+1 vote
rellik_pt
rellik_pt

ok thanks for the reply just ban that member, bot or whatever them

Reply Good karma Bad karma+1 vote
Thanoshld
Thanoshld

It's obviously a member-spam,@rellik_pt.The last addition about reply and control your messages by moddb, is only there because you probably read it through your email provider.I had the same message too, btw.
And good to know how these things work Greg, very interesting!I am looking forward to the changes, though indeed, Moddb is quick enough as it is.But still :)

Reply Good karma Bad karma+1 vote
Cameron:D
Cameron:D

That is some epic CPU usage you've got there :|

Reply Good karma Bad karma+1 vote
Awesome_ninja
Awesome_ninja

any future possibilies to keep track of your comments ?? =,(... I am following so much mods, and when I post a quesion, I can almost never remember for which mod it was, why don't you add a system similar to the one on Youtube? It will be so useful =) !
/me writing down where I posted this comment to check everyday if someone replied...

Reply Good karma Bad karma+2 votes
NullSoldier
NullSoldier

I agree, ModDB needs to revamp the comment system. Comments are currently a second hand citizen when they should not be. Comments now a day are transforming from comments to conversations and I wish there was some way to track comments as well as replies to comments.

Reply Good karma Bad karma+1 vote
Druza
Druza

Good stuff.

Reply Good karma Bad karma+1 vote
jjawinte
jjawinte

I really appreciate you guys taking the time to share this info and enlightening us all a bit more about the management and growth of the site. Thanks for adding the all to rare " human to human " communication element that has made this my favorite gaming community site.

Reply Good karma Bad karma+1 vote
Nilghai
Nilghai

keep up the good work :O

Reply Good karma Bad karma+1 vote
Kasplatt
Kasplatt

zomg its over 130%!

Reply Good karma Bad karma+1 vote
DLCCorrupt
DLCCorrupt

Wow, this upgrade will help a lot!

Reply Good karma Bad karma+1 vote
Guest
Guest

This comment is currently awaiting admin approval, join now to view.

Post a comment
Sign in or join with:

Only registered members can share their thoughts. So come on! Join the community today (totally free - or sign in with your social account on the right) and join in the conversation.