REST ASSURED

I'm proud to announce a project that has been in the works in Opera's QA group for quite some time. It is a tool called MAMA (Metadata Analysis and Mining Application). MAMA was created to help us improve Opera by finding real world sites that we could test with. MAMA allows us to find any combination of CSS, Script and markup factors that we desire. In a sense, it is a search engine for Web page structures instead of content.

MAMA has been tremendously helpful in testing Opera and measuring the popularity of technologies both new and old. One of MAMA's strengths is correlation between different issues. Want to find examples of a "tty" CSS media type that also use the "@import" syntax? Fine. Documents that have over 1,000 inline images or XML documents that use both Flash and external CSS? MAMA can find them.

We know that MAMA is useful, but we think it will be very useful to others too:

• Browser manufacturers and others can use MAMA data on the popularity of widely used technologies to prioritize bugs and justify adding support for new technology to in-progress releases.
• Standards bodies can use the data to measure the success and adoption rates of various technologies.
• Web developers can use the same data to justify support of various technologies in their work.
• It can provide real-world, practical samples of the Web developer's "art", for inspiration and instruction.

MAMA has a side effect that is also one of its most beneficial features: it gives Web authors a voice to the browser and standards makers by documenting actual practice.

MAMA's results for your consideration

In MAMA's introduction, and all the files that link from it, you'll find data that has been pulled from MAMA so far. We lead off MAMA's debut with a full study of markup validation. Along with this, a much shorter, condensed validation article covers some of the broad points of MAMA's validation findings.

Quick links to MAMA's first results:

• Introduction (this is the main starting point)
• Shiny and fancy press release
• Markup validation (full version)
• Markup validation (brief version)
• Key findings
• "The average web page"

...And this is just the initial phase! In the coming weeks, a number of other articles about MAMA's findings and statistics on other Web page topics will be released. Our eventual goal is to make an interface to the MAMA data directly available at some point, but that availability will be phased in gradually when it is ready. For now, these statistics are a way to gauge and generate interest. These articles should also help spur a dialog in guiding MAMA's future collection goals - I find that the more feature requests MAMA gets, the more it improves.

Please send your comments and feedback our way, either directly on dev.opera.com's forums or here on the Opera QA blog.

1. A functional test suite.
   
2. A unit test suite.
   
3. Our beloved continuous integration server, that executes the test suite at every commit and shows the results to the whole team.
   

Though Opera Software's Quality Assurance (QA) department works tirelessly to find problems that affect end-users (that's you), it's impossible to test all possible configurations and to find all possible problems. Thus, we rely on end-users to help find the problems we've missed.

There are several methods end-users can use to communicate their feedback to us:

• Opera Software's Bug Tracking System (BTS)
• Opera Community Forums and newsgroups
• Report a Site Problem
• Blog Comments

The BTS is the best method when end-users run into issues they believe are bugs and they want them fixed. QA employees at Opera Software spend much of their time in the BTS, filing and analyzing bug reports, communicating with developers, and verifying bug fixes. The system makes it easy to track issues and compile relevant information, so any developer viewing a bug report can quickly understand the issue and spend time fixing it rather than analyzing it and gathering information.

In most cases, the bug reports we receive have all the required information, so there is no reason to contact bug reporters. In a small number of cases, we need additional information, so the bug reporter is contacted through the BTS.

The Opera Community Forums and newsgroups are a great place for end-users to discuss issues and gather information before filing bug reports. The forums and newsgroups are not an official way of contacting Opera's QA dept., though we do frequent both the forums and newsgroups. Thus, there's no guarantee that an Opera employee will see your post, let alone act upon it and follow-up internally. Developers generally do not visit the forums and newsgroups, as they're busy coding!

Under the Help menu on Opera for Desktop, users can report problems with the site they're currently viewing and provide a short description of the problem. This information is sent directly to Opera where it's collated in a database. We use this information to augment bug reports by indicating the frequency of problem reports. Reporting a site problem this way does not submit a bug report. Users comfortable reporting bugs should use the BTS instead.

Many of our development teams maintain blogs to communicate with end-users, such as the Desktop Team blog, Mac Team blog, Core Team blog, Opera Mobile blog, and Opera Mini blog. Though these blogs do allow comments, blog comments are a suboptimal means of leaving feedback on a software release. Instead, forums or newsgroups should be used for discussing issues and the BTS should be used for submitting bug reports.

Your feedback is important, so please take a moment to think about which method is most relevant for the feedback you want to provide.

• The team focuses on avoiding and fixing regressions due to clear and quick feedback - less "getting out of the door quickly" with more stability.
  
• The team has confidence in the code - more refactoring or rewriting of suboptimal modules.
  
• Certain processes like deployment are streamlined - easier, clearer and more reliable.