Slightly ajar

There are often cases when web designers wish that browsers support more web standards or fix that pressing bug they always come across. Just how difficult can it be to support all of CSS2.1 or XHTML 1.1 or such? It can't be that difficult, right? Well, unfortunately it often is. I've just been thinking about the many issues we face as an alternative browser, and I though I'd share a little look into what we deal with when producing our rendering engine. This excludes everything else, such as the interface and features of the browser.

Types of issues

There are three broad categories of bugs that we can face when developing a rendering engine:

1. Browser issues
2. Web site issues (known as web or otw bugs at Opera)
3. Specification bugs

Browser issues are bugs and issues that are caused by our browser working incorrectly according to the specification or the de-facto-standard. These are the issues that Opera has to fix in order to be a fully standards compliant browser or work in major web sites. Web site issues are where a web site has bugs due to not following the spec correctly, or deliberately targeting our browser. Specification bugs are where the official spec has issues, such as specifying something that doesn't work in the real world or using vague wording that can be interpreted differently by different browsers. As can be seen, there is more than just following the spec and expecting all sites to work. It is not just this simple however, as each broad type of issue has a number of sub issues types.

Browser issues

The first type of browser issue is where we lack support for a specific part of a standard (such as border-radius in CSS3 for example). This is the part that many developers clamour for, and a very important part of developing a browser. Then there are browser specific extensions. These are generally something we don't want to add, such as ActiveX, but there are cases when it is important due to many sites using that feature or there being no standards based equivalent. Designmode, ContentEditable and xmlHttpRequest are examples of these, even if they are becoming official standards post implementation. These are often not trivial to add as there needs to be a lot of reverse engineering to work out how they should work. As they are not official specs they often don't have a spec to base the behaviour on or public test cases that can be used in the QA process. There is also the question of if we want to be bug compatible with the browser that created the feature in question, even if the quirk doesn't make sense. Added to this, there is the matter of standards and quirks mode, where a feature can act differently depending on what doctype the developer uses.

After missing features there are bugs in already implemented features. Browsers are complex pieces of software, so all have many bugs. The bugs that most people complain about is when a browser doesn't behave like the spec (or other browsers) behave. These are obviously also very important to get fixed as they cause issues that need to be worked around by developers, or leave the site working incorrectly. The bugs people often forget about are the bugs in error handling, where the browser would work correctly if the code was following the standard, but most people don't read the spec and can write invalid code. It is very common that sites don't validate and don't even use a doctype, so resort to the less specified and more murky world of quirks mode, where it can be a nightmare to work out what the browser should handle the content. This is one of the areas that HTML5 looks to specify better. If the error handling is different to what the spec states (if it is defined there) or Opera behaves differently to other browsers then this could be seen in the long term as a Opera bug that needs to be fixed. In the short term Open the Web can be used to contact the site and suggest the standards compliant way to do what was intended (such as include missing quote or whatever). This benefits the site in question as it is more likely to work on more browsers than relying on error handling, which is inconsistent at best. These two bug types can be broken down into two more sections; those that affect a lot of sites or very popular sites, and those that affect just a few sites or very minor sites. It is obvious that we have to concentrate on fixing issues that affect the most users and developers, so these get higher priority. These can often take a higher priority than adding new standards support, and take just as much work. If we have a bug in something like Google.com, even if they don't write to spec, it will affect many millions of users many times a day.

Web Site issues

Then there are web site issues. This is my area of work. No matter how well we follow standards, fix bugs and have have perfect error fallback these issues will cause problems that we can't fix without external help. There are a number of different types of issues in this category, some of which overlap with browser bugs hi-lighted previously. Web site bugs can be where the code breaks the specification. This is where we do what is expected, but the author of the site didn't specify exactly what they meant correctly. This maybe because they just tested in one or two browsers and assumed that if they work there then they wrote the code correctly, when instead they are taking advantage of a browser bug in another browser accidentally. This can often even when testing in two browsers, as one browser may get a different code path (a IE codepath and a standards codepath) or more than one browser implemented the same bugs (such as the event listener bug that both Safari and Firefox had with capturing events). I'd advise to test in at least two out of Opera, Safari and Firefox, at least after major changes or implementing complex JavaScript. If the results differ then a bug has been hit. The next bug type is where Opera's error fallbakc is incorrect or inconsistent. This overlaps with the same issue in Opera bugs, but can be handled by Open the Web if there is a feasible correct solution that still works in the other browsers. Then there are two types of bugs relating to browser sniffing.object detection. The first is where Opera gets sent different code to other browsers and thus breaks when the same code as sent to another browser would work correctly. This issue is very common, especially in old JavaScript files where a very old Opera bugs is being worked around, or the author is just being malicious (believe it or not this seems to happen). Object detection can be a problem as a script can detect one function such as document.all and assume the browser is a different browser (IE in this case) or uses other functions that were not tested, such as using VBScript, ActiveX or CSS filters. The next type of browser sniffing issues are when a site just blocks Opera, or give a browser warning before entering the site. This can be commonly found on Google properties for example.

The next type of web bug is when a site uses either a vendor specific extension to a standard (such as IE or Mozilla DOM extensions) or vendor specific technology (such as ActiveX or VBScript). This sometimes uses browser sniffing and can be included above, but often just is used for all browsers. For example, many video/music sites don't work as they need ActiveX to include the Media Player plug-in that uses DRM, and doesn't exist as a Netscape API plug-in. As mentioned above, for vedor specific extension they can be added into the browser engine if there is a need for this, but that is long term and not always desirable. If there is a standards based solution then open the Web will contact the site and suggest the alternative way to do the same thing. Sometimes there is other solution and the feature has to be added or the site has to stay broken. For vendor specific technology such as ActiveX there is mot often no way to add that technology, without unreasonable amount of work, or restrictions. These are often a lost hope until the site in question redesigns. These issues can also be broken down into issues with top sites and with the long tail. Again top sites will always get most priority as they affect the most users.

It isn't strictly true that Opera can't fix these issues without outside help. We have user agent spoofing for browser sniffing issues, and browser JavaScript to patch site bugs and object detection bugs. These will only be used for popular sites or 3rd party scripts, as there can't be too many patches included, and are quite fragile to site changes, so can only be a stop gap solution. There are two major issues with these techniques. The first is that it causes problems when a developer is testing the site and it isn't behaving as expected, or the patch/spoof isn't applied due to using a different domain for the test server. The latter is that with spoofing, Opera will disappear from site statistics in most cases, making it look like Opera isn't worth supporting.

Specification bugs

The final type of bugs are specification bugs. The w3C, or whoever writes the spec in question, have to be contacted about these issues, and everyone has to agree what is the correct way that the spec should be written and the feature implemented. If Opera ends up on the losing side of a spec change then it has to fix the bug to make it work like the new spec and other browsers. This will render older versions incompatible with the new spec. New versions of a spec can take a long time to be accepted, so these issues can be long term, and with no clear right way until the spec is changed, it is difficult for web designers to make something work cross borwser when using the feature in question.

As can be seen there are many types of different bugs and issues and all this has to be taken into consideration when developing the next version of the rendering engine and what to focus on. Real world compatibility or standards compatibility, adding xyz or fixing that nagging table error fallback bug? Then this can be all multiplied by the number of technologies there are to support in a modern browser, from CSS1,2.1,3, HTML4 and 5, XHTML 1 and 2, XForms, XPath, Web Forms 2, SVG 1.1, client side XSLT, JavaScript, EV Certificates etc. Then there is the rest of the browser wrapped around the rendering engine…

One of the things we've been focused on is increasing our real world compatibility with Kestrel, so I'm very much looking forward to its release. That doesn't mean we've stopped adding new standards though, as I've hi-lighted previously and we'll hopefully be able to add more before release time permitting.

There are various reasons why web sites do not work in browsers such as Opera. Generally they fall into four categories; there is a bug in the browser, a bug in the standards specification, a feature has not been implemented yet, or there is a bug in the site. Although Open the Web to some extent deals with all of these, the later is what we deal with the most, and is what we can mainly help site authors with. These areas are not black and white however and certain issues could be pigeon holed differently depending on interpretation. I'll look at some common web site bugs in this post that effect Opera the most.

Browser sniffing

This is a term for detecting what browser is requesting the content, and is commonly done either server side or in JavaScript/ECMAScript. After detecting the browser different content is served to it. Opera often has problems with this as sites will use it to either block us entirely, send us broken content or forget about us entirely and send no content. There are varying degrees of blocking a browser. The worst will not let the user in at all unless they know how to spoof the user agent string, while others are kinder and warn that the browser may not be supported but you can try anyway. Not all browser sniffing is bad however. It can be useful when there is a known bug in a browser, so that different content is sent that works around the bug. Bugs generally get fixed eventually, so if you take this approach it is best to also detect the version number of the browser that has the bug and only send the adjusted content to that version. That way if the bug is fixed in the next release, your fix will not break the browser.

Object detection

This is generally a better approach than browser sniffing and should be encouraged. With object detection, a function is tested to see if a browser supports it. If it does then you can safely use it, otherwise you can use a known work around. This way a new version of a browser will automatically work with a site once they implement the missing functionality that was tested. Although object detection can do a lot of good, it can also break sites if used wrong. A common way of using it wrongly is to use it as if it is browser sniffing and saying the JavaScript equivalent of If supports document.all browser is IE. Although document.all is an IE extension this assumption falls flat on it's back because Opera also supports it. Object detection should never be used to try to detect what browser is being used. Often the document.all detection is used to then send the browser further IE only code such as CSS filters. As no object detection is used on this because it is assumed we are IE then Opera breaks.

addEventListener

addEventListener is included in the W3C DOM Level 2, and Opera supports it so there should be no problem right? Wrong. The third parameter of this method is useCapture, which specifies if it is a capturing or none capturing event listener. Unfortunately both Firefox and Safari have a bug with this where they treat it as a non capturing event listener no matter if the parameter is true or false. More information about this can be found at this blog post. This causes Opera problems because some scripts that have been reused a number of times use capturing event listeners instead of none capturing, and as it works in other browsers they do not noticed they used the wrong one. This is very easy to fix however by switching the true to false. [UPDATE} Apple are updating their implementation to correct this bug, and it should work the same as Opera on the latest builds of WebKit. This will be good news when this is shipped in Mac OS and Safari. Hopefully Mozilla will follow soon.

position: absolute VS position: relative

IE has a bug where in CSS absolute and relative positioning are treated as the same. Instead absolute positioning should be taken out of the flow of the layout while relative is relative to the containing item. As both seem to do the same in iE, for developers that only test in iE they do not notice the difference and often use the wrong one, meaning content that should be aligned somewhere in the document tends to be stuck at the top left hand corner of the screen in Opera, Safari, Firefox et al. Again this is very easy for the author to fix by swapping the values.

None standard code

This is often used together with browser sniffing or object detection. Browsers such as IE or Netscape in the early days often develop their own none standard code such as Active-X or extensions to the DOM. As only one or limited browsers support it then sites that make use of it will only work in a limited number of browsers. Active-X is the biggest problem as it is IE and Windows only, not to mention a security issue. Not all none standard code ends up bad however. The object that almost single handily started the AJAX revolution, XMLHttpRequest, started life as a IE 5 only extension. Other browsers saw how useful it was, mainly after the likes of Google started using it, and now it has become a de-facto standard if not a real W3C one. As it is so widely used and implemented it is likely that it will be standardised in a W3C specification.

Plug-in mistakes with Flash

I almost missed this one off the list. To include plug-ins such as flash on a page, you have to include an embed tag wrapped in a object tag, so all browsers know how to handle it. It is common however that sites do not give the same parameters in the ibject tag as elements in the embed tag. This means that browsers such as Opera and Firefox that use the embed tag display the plug-in differently to browsers that use object, such as IE. Common mistakes with this are updating the url of a plug-in on the object tag, but forgetting to change the embed tag and specifying different back ground colours. By far the biggest mistake though is missing wmode="transparent" off the embed tag when i is needed. wmode sets the windowing mode of the plug-in. This is especially needed when any JavaScript is going to display over the plug-in area, such as a fly out menu. Without the wmode set to transparent the menu displays behind the plug-in and can't be seen, rendering the site unusable in many cases. It is also useful if your flash content is not square and you want the content under to show through.

Although there are many other issues, these are the most common I come across day to day. If you are a web author, you can minimise these issues by validating your (X)HTML and CSS to ensure valid code is used, check the JavaScript console to make sure there are no errors there, do not use browser sniffing unless needed, use object detection throughout your JavaScript to make sure a browser supports what you are trying to do and finally test in the major browser engines. I'd recommend to test in the following browsers:

• Internet Explorer
• Opera (Presto)
• Safari (WebKit)
• Konqueror (KHTML/KJS)
• Firefox (Gecko)

Testing in all of these, depending if you have access to each of the platforms, will ensure your site works in all the major engines. While WebKit and KHTML are not the same, they are based off each other, so if you do not have access to a Mac or Linux then testing in either/or may suffice. If you have time you should also test without plug-ins and javaScript enabled to make sure your site has fallbacks and remains accessible. Testing on mobiles is becoming increasingly important but not everyone has access to a phone running a browser. In this case Opera's small screen mode (View -> small screen) is useful as it shows how our mobile browser would render the page. This takes handheld stylesheets into account if one is specified. This makes developing mobile friendly code easier and faster, and remember the same engine s used on the Nintendo DS Browser.