Skip navigation.

My Opera News

Behind the scenes at My Opera

November 2009

( Monthly archive )

Back to our primary master database

By Cosimo Streppone. Thursday, 19. November 2009, 18:09:25

, , , ...

My Opera is now back to our more powerful primary master database.
In fact, since the switch we have been running on our secondary (as primary :smile:
Now our old primary, that became secondary, is the primary again. :smile:

We officially switched part of the My Opera database to InnoDB. This has been on the wishlist for such a long time... Now it's finally done. But it's not over. We need to proceed the conversion for all our slaves. This will happen "in the background"...

The failover procedure was really smooth. We're getting better at it, and we plan to automate it completely. I'm not sure if we will ever achieve this, but it doesn't seems to be really hard.
We already have a switch-master script that we can launch against each slave database, and it will safely switch the master db, checking that everything is in order, and it worked quite well today.

Someone the other day on this blog asked about Postgres. I really like Pg, I used it for so many years. MySQL can be good too. We'll see... :smile:

9 comments

My Opera now running on a master to master replication setup

By Cosimo Streppone. Monday, 16. November 2009, 21:25:11

, , , ...

The scheduled database maintenance was completed on Saturday. Well, it wasn't exactly maintenance, but rather an important change of setup. Now My Opera is running on a master to master replication setup, instead of the classic master-slave that has been running for some years now.

There's at least 2 good reasons why we chose that setup:
  • it allows for database master maintenance with practically no downtime, since you can have the site run on your secondary master, while the primary is taken offline. When maintenance is over, you can switch back to the primary.
  • in case of disaster or failures to the primary master, the secondary can kick in (manually for now) in a rather short time. Last time we had a primary master db crash, it took us something around 4-6 hours to be back online.


So, currently My Opera is running on the secondary master database, while the primary is being prepared. "Being prepared" in this case means that we're converting some of our heavy duty tables to the InnoDB storage engine. This was really needed because MyISAM table-level locking poses some hard limits on concurrency and scalability of your database, even if it's very fast. Add that we have more user activity, APIs and services than ever before, and you have the whole picture.

Of course, all this work should also improve raw performance of the site. Once we get back to our primary master server, which we will probably do in the next few days, the site will be definitely faster. Not ludicrous speed yet, but faster :smile:

19 comments

Scheduled downtime for database maintainance

By Cosimo Streppone. Friday, 13. November 2009, 11:01:42

, , ,

We are planning a database maintainance operation today from 19:00 UTC, 20:00 CET. Estimated downtime should be around 1 hour.

We tested this already on other db systems and we definitely saw good results, so cross your fingers.

We're looking forward to have a faster My Opera.

EDIT (2009/11/14 00:13:37): There has been no downtime so far.
We're continuing the maintainance operation tomorrow. Still 1 hour of estimated downtime. That will probably happen during the afternoon or early evening.

25 comments