Slightly ajar

There are often cases when web designers wish that browsers support more web standards or fix that pressing bug they always come across. Just how difficult can it be to support all of CSS2.1 or XHTML 1.1 or such? It can't be that difficult, right? Well, unfortunately it often is. I've just been thinking about the many issues we face as an alternative browser, and I though I'd share a little look into what we deal with when producing our rendering engine. This excludes everything else, such as the interface and features of the browser.

Types of issues

There are three broad categories of bugs that we can face when developing a rendering engine:

1. Browser issues
2. Web site issues (known as web or otw bugs at Opera)
3. Specification bugs

Browser issues are bugs and issues that are caused by our browser working incorrectly according to the specification or the de-facto-standard. These are the issues that Opera has to fix in order to be a fully standards compliant browser or work in major web sites. Web site issues are where a web site has bugs due to not following the spec correctly, or deliberately targeting our browser. Specification bugs are where the official spec has issues, such as specifying something that doesn't work in the real world or using vague wording that can be interpreted differently by different browsers. As can be seen, there is more than just following the spec and expecting all sites to work. It is not just this simple however, as each broad type of issue has a number of sub issues types.

Browser issues

The first type of browser issue is where we lack support for a specific part of a standard (such as border-radius in CSS3 for example). This is the part that many developers clamour for, and a very important part of developing a browser. Then there are browser specific extensions. These are generally something we don't want to add, such as ActiveX, but there are cases when it is important due to many sites using that feature or there being no standards based equivalent. Designmode, ContentEditable and xmlHttpRequest are examples of these, even if they are becoming official standards post implementation. These are often not trivial to add as there needs to be a lot of reverse engineering to work out how they should work. As they are not official specs they often don't have a spec to base the behaviour on or public test cases that can be used in the QA process. There is also the question of if we want to be bug compatible with the browser that created the feature in question, even if the quirk doesn't make sense. Added to this, there is the matter of standards and quirks mode, where a feature can act differently depending on what doctype the developer uses.

After missing features there are bugs in already implemented features. Browsers are complex pieces of software, so all have many bugs. The bugs that most people complain about is when a browser doesn't behave like the spec (or other browsers) behave. These are obviously also very important to get fixed as they cause issues that need to be worked around by developers, or leave the site working incorrectly. The bugs people often forget about are the bugs in error handling, where the browser would work correctly if the code was following the standard, but most people don't read the spec and can write invalid code. It is very common that sites don't validate and don't even use a doctype, so resort to the less specified and more murky world of quirks mode, where it can be a nightmare to work out what the browser should handle the content. This is one of the areas that HTML5 looks to specify better. If the error handling is different to what the spec states (if it is defined there) or Opera behaves differently to other browsers then this could be seen in the long term as a Opera bug that needs to be fixed. In the short term Open the Web can be used to contact the site and suggest the standards compliant way to do what was intended (such as include missing quote or whatever). This benefits the site in question as it is more likely to work on more browsers than relying on error handling, which is inconsistent at best. These two bug types can be broken down into two more sections; those that affect a lot of sites or very popular sites, and those that affect just a few sites or very minor sites. It is obvious that we have to concentrate on fixing issues that affect the most users and developers, so these get higher priority. These can often take a higher priority than adding new standards support, and take just as much work. If we have a bug in something like Google.com, even if they don't write to spec, it will affect many millions of users many times a day.

Web Site issues

Then there are web site issues. This is my area of work. No matter how well we follow standards, fix bugs and have have perfect error fallback these issues will cause problems that we can't fix without external help. There are a number of different types of issues in this category, some of which overlap with browser bugs hi-lighted previously. Web site bugs can be where the code breaks the specification. This is where we do what is expected, but the author of the site didn't specify exactly what they meant correctly. This maybe because they just tested in one or two browsers and assumed that if they work there then they wrote the code correctly, when instead they are taking advantage of a browser bug in another browser accidentally. This can often even when testing in two browsers, as one browser may get a different code path (a IE codepath and a standards codepath) or more than one browser implemented the same bugs (such as the event listener bug that both Safari and Firefox had with capturing events). I'd advise to test in at least two out of Opera, Safari and Firefox, at least after major changes or implementing complex JavaScript. If the results differ then a bug has been hit. The next bug type is where Opera's error fallbakc is incorrect or inconsistent. This overlaps with the same issue in Opera bugs, but can be handled by Open the Web if there is a feasible correct solution that still works in the other browsers. Then there are two types of bugs relating to browser sniffing.object detection. The first is where Opera gets sent different code to other browsers and thus breaks when the same code as sent to another browser would work correctly. This issue is very common, especially in old JavaScript files where a very old Opera bugs is being worked around, or the author is just being malicious (believe it or not this seems to happen). Object detection can be a problem as a script can detect one function such as document.all and assume the browser is a different browser (IE in this case) or uses other functions that were not tested, such as using VBScript, ActiveX or CSS filters. The next type of browser sniffing issues are when a site just blocks Opera, or give a browser warning before entering the site. This can be commonly found on Google properties for example.

The next type of web bug is when a site uses either a vendor specific extension to a standard (such as IE or Mozilla DOM extensions) or vendor specific technology (such as ActiveX or VBScript). This sometimes uses browser sniffing and can be included above, but often just is used for all browsers. For example, many video/music sites don't work as they need ActiveX to include the Media Player plug-in that uses DRM, and doesn't exist as a Netscape API plug-in. As mentioned above, for vedor specific extension they can be added into the browser engine if there is a need for this, but that is long term and not always desirable. If there is a standards based solution then open the Web will contact the site and suggest the alternative way to do the same thing. Sometimes there is other solution and the feature has to be added or the site has to stay broken. For vendor specific technology such as ActiveX there is mot often no way to add that technology, without unreasonable amount of work, or restrictions. These are often a lost hope until the site in question redesigns. These issues can also be broken down into issues with top sites and with the long tail. Again top sites will always get most priority as they affect the most users.

It isn't strictly true that Opera can't fix these issues without outside help. We have user agent spoofing for browser sniffing issues, and browser JavaScript to patch site bugs and object detection bugs. These will only be used for popular sites or 3rd party scripts, as there can't be too many patches included, and are quite fragile to site changes, so can only be a stop gap solution. There are two major issues with these techniques. The first is that it causes problems when a developer is testing the site and it isn't behaving as expected, or the patch/spoof isn't applied due to using a different domain for the test server. The latter is that with spoofing, Opera will disappear from site statistics in most cases, making it look like Opera isn't worth supporting.

Specification bugs

The final type of bugs are specification bugs. The w3C, or whoever writes the spec in question, have to be contacted about these issues, and everyone has to agree what is the correct way that the spec should be written and the feature implemented. If Opera ends up on the losing side of a spec change then it has to fix the bug to make it work like the new spec and other browsers. This will render older versions incompatible with the new spec. New versions of a spec can take a long time to be accepted, so these issues can be long term, and with no clear right way until the spec is changed, it is difficult for web designers to make something work cross borwser when using the feature in question.

As can be seen there are many types of different bugs and issues and all this has to be taken into consideration when developing the next version of the rendering engine and what to focus on. Real world compatibility or standards compatibility, adding xyz or fixing that nagging table error fallback bug? Then this can be all multiplied by the number of technologies there are to support in a modern browser, from CSS1,2.1,3, HTML4 and 5, XHTML 1 and 2, XForms, XPath, Web Forms 2, SVG 1.1, client side XSLT, JavaScript, EV Certificates etc. Then there is the rest of the browser wrapped around the rendering engine…

One of the things we've been focused on is increasing our real world compatibility with Kestrel, so I'm very much looking forward to its release. That doesn't mean we've stopped adding new standards though, as I've hi-lighted previously and we'll hopefully be able to add more before release time permitting.