Skip navigation.

My Opera News

Behind the scenes at My Opera

Static resources on lighttpd

, , , ,

It's been a while since we deployed the first static server for My Opera.
It was a really necessary step, because we had, and we continue to have, lots of static resources to be served.
A while back, most of these resources were served by applications, causing much more load on servers than necessary.
Now most of the heaviest ones have been already moved to the static servers.

Some months ago we also added support for partitioning of the user resources in our storage software layer. That means making sure that if we have the need to scale serving of resources on different machines, we can either replicate the entire content on different ones, or split the content over 2 or more machines. That part worked nicely so far. That's why at some point there was a static.myopera.com and static02.myopera.com.

Now there's also static03.myopera.com :smile:

We changed our setup again, transparently (for you users), to consolidate the previously recycled/temporary hardware into a new shiny machine with more disk space. On this machine, we installed lighttpd instead of our usual Apache setup. We wanted to try out this software. For us it was the first time we tried it on a production setup.

Edoardo played with it for a while, prepared the setup and installed it on static03. As of today, it has been running perfectly for nearly 1 month with a really low load, and peaks of 250 accesses per second. It's serving around 14M hits per day for avatars, user pictures, skin thumbnails, etc...

Dragonfruit releaseScheduled downtime for database maintainance

Comments

rkyrylych 29. October 2009, 21:20

:cool:

remcolanting 29. October 2009, 21:24

lighttpd <3

Oh, and faster my.opera.com <3

Muttsfan 29. October 2009, 21:27

:yes:

Chas4 29. October 2009, 21:32

cstrep 29. October 2009, 21:38

@remco: well, today I guess it wasn't so fast :-|
We're getting massive amounts of traffic. Working on counter measures. Stay tuned... :smile:

Sterkrig 29. October 2009, 21:46

Why not nginx? (-:E

Chas4 29. October 2009, 22:00

Originally posted by cstrep:

Stay tuned.


Staying tuned

davidtsunamy 29. October 2009, 22:04

:headbang:

Furie 30. October 2009, 02:06

Is there a way to check if a certain user's avatar is served from there?

philry4n 30. October 2009, 04:38

Originally posted by cstrep:

@remco: well, today I guess it wasn't so fast :-|
We're getting massive amounts of traffic. Working on counter measures. Stay tuned...


What happened? something got reddited, twittered, or digged?

NoobSaibot 30. October 2009, 10:58

i'd have voted for nginx as well :wink:

Leevi 30. October 2009, 11:26

@Furie use 'Open Image' and you can see it from the url, everyone's avatar here seems to be on static03

@philry4n Opera 10.01 came out yesterday, that might have something to do with it :wink:

Furie 30. October 2009, 12:21

Hmmm, I see now, thanks. My theory is out though. :irked:

cstrep 30. October 2009, 14:08

We're keeping an eye on Nginx too.

Porting the config files is not so straight forward as with Lighttpd, plus the distro packages are too old, as nginx moves forward (and needs to be updated) more quickly. At least this is my impression after following its mailing list for about 3/4 months.

Sterkrig 30. October 2009, 14:39

Originally posted by cstrep:

Porting the config files is not so straight forward as with Lighttpd


When I chose between lighttpd and nginx, configs of latter one seemed much more comprehendible to me (-:E
That's just personal opinion, of course. I just asked why (-:E

rafaelluik 30. October 2009, 17:28

I feel My.Opera running faster. :smile:

53north 31. October 2009, 07:54

Now to get the bloggers to optimize their images before using a 1Mb png for a user pic or blog bg..
=o}

cstrep 31. October 2009, 08:02

@53north: they are already scaled in several different formats, as you can probably deduct from the URLs.

53north 7. November 2009, 22:58

They still crash browsers this side..
=o}
user Emolasleira's page is real work..

Write a comment

You must be logged in to write a comment. If you're not a registered member, please sign up.