Skip navigation.

Sign up | Lost password? | Help

My Opera News

Behind the scenes at My Opera

Yet another performance optimization round

By Cosimo Streppone. Wednesday, 18. March 2009, 15:52:50

, , , , ,

After we cut down the database size by an impressive 22 Gb yesterday (and we had our share of problems afterwards, as you probably noticed...), we managed to get some performance-related fixes out live.

According to our monitoring tools, all the servers are running faster and with lot less load than ever before. Of course, it's not going to last long, because traffic is slowly but continuously increasing. At least it should give us some more time to relax a bit (read: dedicate only to development stuff...).

For the curious among you, I've attached a screenshot of one of our own monitoring tools. A pity you can't see the realtime updates. This is taken at peak time, and it shows load, active threads and replication status for each machine. It shows most of the My Opera backend machines:

where green stuff is OK, while orange or red is not.

To add more fun, one of our main database servers has been repeatedly crashing during the weekends (mostly friday and saturday nights, super-fun!). Most of the time My Opera has been up and running without being affected. Today this server was updated with a brand new kernel and disk driver version. It shouldn't crash anymore. Now we're pushing it at full speed. Let's see...

Happy birthdays!Disappearing user pictures

Comments

MConor 18. March 2009, 15:58

Good work guys :up: The Online system is still a little flaky, but overrall much improved :up:

SouthernCross 18. March 2009, 15:58

:up:

Aleksander 18. March 2009, 15:59

I can has moar speed?

nicomen 18. March 2009, 16:14

:beer:

MConor 18. March 2009, 16:18

Just looking at the picture... Would it not make more sense to put the same strain on all the databases, you know, divert to some of the lesser ones? It doesn't make sense to have on at 15% and one at 65%. Just speculating here, I don't know much.

Once again, Good work :cheers:

nicomen 18. March 2009, 16:27

Conor M: percentage there is actually disk space left ;-) And the different DBs also differ in performance/hardware/setup btw. And they are load balanced in different ways.

MConor 18. March 2009, 16:41

Thanks Nico :yes: I was just shooting in the dark anyway :lol:

Tamil 18. March 2009, 16:47

:up:

drlaunch 18. March 2009, 18:30

Are you using any virtualisation between the servers? Or does the load balancing eliminate the need for it?

theoddbod 18. March 2009, 19:58

:cool:

Chas4 19. March 2009, 03:15

:up:

how many pizza this time?

cstrep 19. March 2009, 08:53

Chas4: Actually, we need to buy some pizza. Will file a bug :smile:

drlaunch: we're not using virtualization, just LVS load balancing with TCP handoff (does that sound cool? :-)

xErath 19. March 2009, 10:29

Originally posted by devblog:

After we cut down the database size by an impressive 22 Gb yesterday


OMG ! You never defragmented the database ? :eek:

nicomen 19. March 2009, 12:07

xErath: after a year or so of migrating we finally got rid of user pictures that were stored in the database. (Don't ask!) :-)

BluePaper 7. June 2009, 10:51

That looks pretty sweet I have to say, I was just thinking of looking into creating my own server monitoring system for a challange. I've no idea which angle to go at it though, what language did you use to develop your system? I was thinking of using Python seeing as it can be used on the web-end via CGI or mod-python as well as it being able to access other system files. Any ideas?

Any way, good to hear your continuing to optimise and improve MyOpera, this blog makes quite a fascinating read :smile:

cstrep 7. June 2009, 19:19

@BluePaper: thanks, we used Perl and Javascript for this. The web application talks with Ajax calls to the Munin daemons running on each machine to extract the needed information.

Write a comment

You must be logged in to write a comment. If you're not a registered member, please sign up.