miscoded

the web is a hack

Subscribe to RSS feed

Sticky post

Introduction

My journal is Opera-related and technical. It will cover the main obstacles we come across when we use Opera on the Web as it is - the standard violations, the browser incompatibilities, the sniffers and faulty scripts. That is the whole mess a poor browser has to make sense of and believe me, Opera is doing a brilliant job.

web browsing as forensic evidence in Norwegian murder case

A very peculiar murder is being investigated by the Norwegian police, known as the "Skøyen homocide" after the Oslo region where it happened. Basically a man was killed with a knife in a garage, he had no known enemies and there is no obvious motive. The police investigation so far has made the police believe that one of the neighbours did it. There is some technical evidence, specialists claim a knife the neighbour owns is "more likely to be the murder weapon than not be the murder weapon", and there are assumptions based on his computing activity on that day. Basically, there is a 10 minute gap of inactivity around the time the police assumes the victim arrived at the garage. The accused explains that he was working from home because his daughter was ill, and any absence from the computer is due to him taking care of her.

I have no inside information here, I have only paid some attention to media reports because the case is so peculiar. (Personally I don't find the police's scenario very convincing. Who would leave their web surfing, walk back to their garage to commit a messy knife murder of a random person, walk back up again and continue surfing only ten minutes later?) But the latest news from one of Norway's most respected daily newspapers reveals some more details about the computer evidence - and it says that the accused even was an Opera user:

(..) between 16.03.48 and 16.13.11 there was no clear evidence of user activity on the computer (..) - In this interval we do have the timestamps that have been a topic as evidence, Holden [police lawyer] points out.

The court debates whether these so called timestamps are created due to user activity or by the computer's software.
(..)
The conclusion from Kripos [special police unit] was that other software than the accused's Opera E-mail client may have created the debated timestamps, but they can not confirm what kind of software may be involved.
(..)
Defence lawyer Petter Nordgreen Sterud (..) says five timestamped files and one cookie are from the interval between 16.03 and 16.13, and he believes that no other good explanations than user activity have been put forward.



Of course, a web browser is absolutely not a reliable witness for user activity. Today's all-singing all-dancing websites can reload bits and pieces at any time, thus cause new cached files without anyone being present at the computer. We don't get any information about what type of files one is dealing with - but the specific reference to the mail client is interesting. If the mail client fetched mail (either at an automated interval or because of a manual "check now" action) it may well have created five new files for downloaded E-mail - but in that case, it should be pretty obvious to the Kripos experts what software was responsible.. If it is about mail client files, the defence should ask whether Kripos checked the interval between mail client timestamps against the interval for automated checks. An unexpected timestamp in the sequence would be a strong indication of user activity.

The cookie may not be significant as evidence either, but it certainly could be, depending on its name, what site served it, and its value. I certainly hope Kripos is professional enough to check where the cookie came from and what sort of process would set it (and I hope the defence lawyers are skilled enough to check whether this has been done). If it is typically set in response to user activity, it could give significant weight to the defence's argument that the accused man was in his flat.

For me, the case raises deeper questions too: how do we explain the details of computing to the public and the authorities? Your aunt might have a very fuzzy interpretation of what a "cookie" is, but one day she may be asked to sit on a jury and listen to computer-related evidence.. Sometimes there is no good substitute for computer literacy..

CodeMirror's autocomplete broken by Opera keyboard accessibility feature

, , ,

Incidentally, just a couple of days after reading and commenting on this blog post regarding current Opera versions' accessibility shortcomings, I find a page that is broken precisely due to one of the accessibility features we do have..

As far as I know, Opera is the only browser that lets users use SELECT elements with several possible choices entirely from the keyboard. I've got a demo here, if you tab into the SELECT element you'll see that you can move focus with up/down arrows and use the space key to create selections. The code is good, old-school HTML - as simple as it gets:

<select multiple size="4">
	<option>A</option>
	<option>B</option>
	<option>C</option>
	<option>D</option>
</select>


CodeMirror.net has an autocomplete-demo - for example try to type 'get' and press ctrl-space in that editor. It's rather clever of them to pop up a dynamically created SELECT element to let you choose from - but unfortunately, they've set select.multiple to true:
    // Build the select widget
    var complete = document.createElement("div");
    complete.className = "completions";
    var sel = complete.appendChild(document.createElement("select"));
    sel.multiple = true;

Hence, if you try to use arrow-down you'll see Opera's pre-selection focus move down, the selection won't change. If you try to use space, the page interprets that as a confirmation of the current selection. It's just not possible to select anything but the first entry by keyboard.

Our best efforts at making <SELECT multiple> keyboard-accessible makes this page inaccessible. It's hard to innovate on the web. Even the subtlest innovations for the best reasons can cause compatibility problems..

In this case it's clearly the site's fault. It is semantically incorrect of them to say that this list is a multi-select when the user is only expected to select one single entry. (Luckily, I expect the coders behind this site to be receptive to arguments based on semantic correctness smile.)

Anyway, being stuck with this choice is obviously a bad thing:
<select>
<option>Accessible</option>
<option>Compatible</option>
</select>

What's wrong with that code? Well, someone forgot to put in the multiple attribute, of course.

anti-semantic ARIA abuse - it just feels so wrong

,

I'm not going to name and shame anyone, but doesn't this stuff look downright stupid?

<span role="checkbox" dir="ltr" class="jfk-checkbox goog-inline-block jfk-checkbox-unchecked" tabindex="0" aria-checked="false"></span>


Web enthusiasts and developers spent years on pondering the semantics of HTML elements, and countless hours on writing software to express and interpret said semantics. A huge library of software for all sorts of platforms and end users embodies the conclusions, in absolutely every major programming language you've heard of.

And nowadays, if you just add some magic aria-foo properties you can stop caring about semantic code without even feeling bad about it?

Somebody will tell me that this is the sort of markup that has been used all along, but now more accessible thanks to aria-*. Give me a break! If a coder or authoring tool author knows enough coding to know about aria-role, s/he should certainly understand the value of semantic markup too!

either too old or too modern (!)

, , ,

A few things about this source code might be amusing, but the funniest thing is that the developer writing it was clear-sighted enough to predict that a web browser "too modern" to deal with his code might one day emerge:
//PopUp Calendar BEGIN--------------
if (navigator.appName != "Netscape")
{
if (document.all) {
	document.writeln("<div id=\"PopUpCalendar\" style=\"position:absolute; left:0px; top:0px; z-index:7; width:200px; height:77px; overflow: visible; visibility: hidden; background-color: #FFFFFF; border: 1px none #000000\" onMouseOver=\"if(ppcTI){clearTimeout(ppcTI);ppcTI=false;}\" onMouseOut=\"ppcTI=setTimeout(\'hideCalendar()\',500)\">");
	document.writeln("<div id=\"monthSelector\" style=\"position:absolute; left:0px; top:0px; z-index:9; width:181px; height:27px; overflow: visible; visibility:inherit\">");}
else if (document.layers) {
	document.writeln("<layer id=\"PopUpCalendar\" pagex=\"0\" pagey=\"0\" width=\"200\" height=\"200\" z-index=\"100\" visibility=\"hide\" bgcolor=\"#FFFFFF\" onMouseOver=\"if(ppcTI){clearTimeout(ppcTI);ppcTI=false;}\" onMouseOut=\"ppcTI=setTimeout('hideCalendar()',500)\">");
	document.writeln("<layer id=\"monthSelector\" left=\"0\" top=\"0\" width=\"181\" height=\"27\" z-index=\"9\" visibility=\"inherit\">");}
else {
	document.writeln("<p><font color=\"#FF0000\"><b>Error ! The current browser is either too old or too modern (usind DOM document structure).</b></font></p>");}
}
	document.write ("<noscript><p><font color=\"#FF0000\"><b>JavaScript is not activated !</b></font></p></noscript>");
The result is the red warning message on this page of an airline I must be too modern to book tickets from: So, actually, simply by switching navigator.appName from "Netscape" to "Opera" a few years back we became modern. So that's a challenge to Chrome, Firefox and IE: you know you want to be as modern as us - what are you waiting for? smile

Don't look to IE anymore..

, ,

I've been looking at a bug reported for the Chinese site 56.com (what's up with China and all those numerical domain names - lucky numbers smile?) Here's a screenshot from a colleague, Opera on the left and Chrome on the right:


Digging a bit, it boils down to a failure in a piece of JavaScript that reads innerHTML of an element and does a regular expression match() that ends up matching nothing. They use a regular expression created from this string:

var sRepeat="<%begin_"+xLev+"[^>]*%>((.|\\n)+?)<%end_"+xLev+"%>"; 


and it's a problem with how the line endings are encoded. The site's code contains typical Windows line endings (newline + carriage returns) but this regular expression assumes that innerHTML will contain newlines and not carriage returns. (This is the \\n part of the regexp. The fix is simply replacing "\\n" with "\\n|\\r".)

Firefox and Chrome return innerHTML with normalized LF line endings. IE and Opera return CRLF in this case, and comments are broken both in IE (8) and Opera for me.

Wait..did I just say that? We break a shiny, mass-market production site because we follow IE closely on a piece of JavaScript IE invented and defined? And IE is broken too??

Wow! The times they are a-changing..

The new Google Maps patch, dissected

, ,

We released a new browser.js file yesterday. It's a pretty interesting release actually..

The Amazon patch is a pretty radical step - we have never launched any patch that embeds a third-party JS library on every page of a global web presence like Amazon. We know it solves some of our problems, and we know it doesn't solve all problems - we'll keep working with Amazon to have more of their server-side browser sniffing fixed. It would be nice if this patch were short-lived wink


But the most interesting patch - or at least the hardest one to come up with - is a workaround for a bug in the Carakan ECMAScript engine that a new Google Maps feature hit. The feature is being rolled out, so some users have seen this problem and some haven't yet. As you see from the screenshot, they are very excited about the new feature and announce it with a small popup. In Opera, that popup refuses to go away again. Using the close button just brings up the small "loading" indication at the top of the screenshot. Using Google Maps with this stubborn box staying on top of everything else is more than a little annoying..

My colleague Kåre did the hard work of reading obfuscated JS to figure out where the problem was. I don't want to know how many hours it took him to boil it down to this compact demo:
var passed = 0;
(function (){
	var foo = 1;
	this.runtest = function (){
		eval('window.ev=function(a){eval(a)};');
		try{
			window.ev('(function(){passed=foo})()')
			}
		catch(e){}
		document.getElementsByTagName('p')[0].firstChild.data = passed?'PASS':'FAIL';
		};
	})();
runtest();


To understand this issue, including why finding a patch at first seemed impossible, we need to know about an esoteric quirk of the eval() function.

Normally, if you're naughty enough to call eval() (which it is better not to use, by the way..) you invoke the window.eval() built-in function. It will evaluate the code you pass to it in the local scope. So, for example doing this:
function(){
  var ok=true;
  eval('alert(ok?"sees local variable":"does not see local variable")');
}

the eval'ed code sees the local variable 'ok'.

However, eval by any other name doesn't smell as sweet - if the name you use to refer to the eval function is not 'eval' the method will change behaviour entirely, and run code in the global scope instead! Doing this:
var ok=false;
function(){
  var ok=true, ev=window.eval;
  ev('alert(ok?"sees local variable":"does not see local variable")');
}

we see that if the reference to the eval method is different, it no longer sees local variables!

This odd behaviour is a known eval quirk which is now standardised in ECMAScript 5.

Regarding the problem, Google Maps seem to have found a way to do indirect eval in a local scope. Sort of. I have no idea why they are doing it this way..

The window.ev('(function(){passed=foo})()') just references another method that will do the actual eval'ing. Doing an indirect eval here would run code in global scope. Hence it should not actually have accesss to the local 'foo' variable.

However, the earlier eval call is a direct eval - so it will run in the local scope:
eval('window.ev=function(a){eval(a)};');
meaning that the function expression here should have closure data that would include the local variable foo.

Hence, when a string of code that refers to 'foo' is passed to the direct eval() inside that ev function, this eval call will run in the local scope of that function expression, and the variable foo should be found in its closure data. And this is where Carakan stumbles: it fails somehow to get the closure data right for a function expression created in an eval() inside another function.

Now, why was this hard to patch? Normally when some method Opera defines has a bug, we can overwrite that method with a fixed one from browser.js. For example, if we find a problem in the DOM appendChild() method we'd just overwrite Node.prototype.appendChild with a working method. However, we can not overwrite window.eval() like that - because we would need to cache a reference to it, and any calls to that reference would be an "indirect" eval and run in the global scope.. This would violate most of Google Maps assumptions and make the site fail entirely.

Here's the patch I eventually came up with:
addPreprocessHandler(  /function\(a\)\{eval\(a\)\}/g, '(function(){return function(a){eval(a)}})());' );


This does a search/replace on Google Maps' source code while we load it, replacing any instances of
function(a){eval(a)}

with
(function(){return function(a){eval(a)}})());

which adds an extra function expression - one more chance for Carakan to record closure data. This double-take does, by a lucky stroke, save the required context.

So, here's hoping most users get the new browser.js before the Google Maps update. I like the speeeed of patching problems before users even get them smile

Calling users who have problems loading GMail in standard mode..

,

Several users are reporting temporary problems loading GMail. The typical symptom is that it hangs before the "loading" progress bar reaches 100%, and the user needs to switch to plain HTML mode or re-load multiple times to see their inbox.

Now, the annoying thing about this problem is that it keeps happening to users but I have never been able to reproduce it myself. We really need to get to the bottom of this, and it seems we really need some help to get there!

As-is, I don't have any real clue to what the problem is. My best guess is that some request doesn't complete or some event isn't sent. Hence, I've prepared a user script - gmail-log.js - that logs XHR requests and important events to the error console when you load GMail.

So, if you are a reasonable computer-literate end user who is currently experiencing this ghostly GMail loading problem, here's how you can help:

  1. Save that script to your computer
  2. Make Opera load it as User JavaScript (remember to enable User JavaScript on HTTPS in opera:config)
  3. Open the Error Console and clear any existing content
  4. Load GMail (confirm that you want to run User JS on HTTPS when the dialog appears). If you see the problem, then..
  5. Copy the error console contents (Ctrl-A, Ctrl-C) and send me by PM or E-mail.
  6. Disable the user script again (it will slow GMail down a little bit, so no point in keeping it active)
  7. Please leave a comment on this blog saying that you've done so. With some luck, 5-10 people submitting logs would give us some interesting clues..


If you're not sure how to do some of that, I suggest you wait a few days and see if enough people contribute smile

Note: while this script does not intend to capture any personal information, I can not guarantee that the log won't contain session IDs or E-mail data. I suggest you don't post it in public.

Thanks a lot in advance to anyone who can shed some light on this!