Skip navigation.

Log in | Sign up

REST ASSURED

Quality rants by Opera QA

Posts tagged with "continuous integration"

Testing My Opera

, , , ...

Here at Opera, we make more things than the desktop browser. Apart from the browser for all the other platforms and Opera Mini, we make some internal systems, some server-side support for the browser, and some public sites. Probably the most well known, non-browser Opera product is My Opera (yes, this very site!). Being such an important product, it's in the hands of very smart people (and me), and it goes through extensive automated testing to catch as many problems as possible, as early as possible.

The automated testing for My Opera is based on three things:

  1. A functional test suite.
  2. A unit test suite.
  3. Our beloved continuous integration server, that executes the test suite at every commit and shows the results to the whole team.


The functional test suite is built with Test::WWW::Mechanize and uses special My Opera testing installations, each one connected to its own private database. That way, we can recreate the database before each test and ensure a stable and reliable environment for the test execution. This is currently the biggest and most mature one, and it has helped a lot in avoiding regressions when refactoring code and fixing bugs.

The unit test suite is a standard Perl test suite that covers the most important modules of My Opera. Again, we use private databases for the tests that do need them, so we ensure a reliable testing environment.

Last, but not least, we have everything hooked up in continuous integration so we have testrun information for every commit. When any test suite catches errors, we are automatically notified, and we can track which changes produced which errors.

All in all, we are quite happy with the automated tests for My Opera, as they give us freedom to work with the code without fear of breaking things without noticing. There is, of course a lot of room for improvement, and we are always tweaking and enhancing it.

Continuous Integration: Team Testing

, , , ...

As you know, Continuous Integration is a set of simple practices that reduces software integration time and problems. It also improves general software quality with little overhead. Just having a testrun per every commit is quite handy for two reasons: early regression finding and easier debugging.

One of the most important virtues of CI is giving the tests a "social dimension". Martin Fowler's "Everyone can see what's happening" is actually one of the most powerful, yet subtle virtues of Continuous Integration. Letting everyone know about the status of the tests is not about assigning blame when something fails, it's about making the tests part of the development process. We wholeheartedly agree with the vision of automated tests that are incorporated upstream into the development process and run on a continuous basis.

What happens when you make the tests part of the process? Several interesting things:

  • The team focuses on avoiding and fixing regressions due to clear and quick feedback - less "getting out of the door quickly" with more stability.
  • The team has confidence in the code - more refactoring or rewriting of suboptimal modules.
  • Certain processes like deployment are streamlined - easier, clearer and more reliable.


We feel that these benefits make Continuous Integration a natural extension to automated tests for development teams, a bit like "the version control system of tests". Here at Opera we are using Continuous Integration for selected server-side projects, notably the Opera Link server and My Opera. It has had a huge and positive impact on both of these projects, providing a crucial role when refactoring and improving code.

If you are not using Continuous Integration in your project yet, you should start now!