miscoded

Working for Opera Software's QA department gives you in-depth perspectives on the web's wild and varied coding practises. I still wasn't prepared for the curious solutions that power the menu on the new Israeli rail website.

The XSLT markup/programming language is widely used to transform one sort of DOM into another - for example turning the DOM of a generic XML file into valid XHTML. Much of the benefit is that you're working on DOM trees - making it hard or impossible to create syntactically invalid pages.

Diving into the source code shows that the JavaScript coders working on the Rail site were asleep during their education's "what's the point of XSLT" lesson. The coding is unbelievable. It's more like an XML parser/serializer stress test than a production site. Now, I don't really know XSLT and trying to debug this confirms my impression that it must be one of the worse programming languages mankind has invented - but the point of this script is to generate HTML with XSLT *string concatenation*?!?? Look at this:

<xsl:value-of select="$attribute-name"/>="<xsl:call-template name="inner-attribute-text-value"><xsl:with-param name="attribute-value" select="$attribute-value"/></xsl:call-template>"

or

<xsl:template name="inner-text-tag-open"><xsl:text disable-output-escaping="yes"><</xsl:text></xsl:template>
<xsl:template name="inner-text-element-close">
<xsl:param name="element-name"/><xsl:call-template name="inner-text-tag-open"/>/<xsl:value-of select="$element-name"/><xsl:call-template name="inner-text-tag-close"/></xsl:template>
<xsl:template name="inner-text-tag-close"><xsl:text disable-output-escaping="yes">></xsl:text></xsl:template>

Yes, all that to create a text node containing e.g.

</div>

in a DOM they will serialize only to parse it again by setting innerHTML on some poor element..

When they in their wisdom chose to generate markup inside text nodes with their XSLT they run into the familiar problem: when is < going to start a tag and when is it going to live in a text node? Hence, < is sometimes escaped as an 'lt' entity to create proper text nodes with HTML source-as-text in them (see for example the instance of

<

in the code above). Now, of course when they set innerHTML they do not want this entity to appear as a literal < so they do some pre-processing: all entities they want to change into proper < and > before setting innerHTML have a comment node next to them:

<!--nwlt-->&lt;TR class="nw-2r"&gt;<!--nwgt--><!--nwlt-->&lt;TD class="nw-2c"&gt;<!--nwgt-->

and their pre-processing is a simple string replace:

sHtml = sHtml.replace(/\<!--nwlt--\>&lt;/g,"<").replace(/&gt;\<!--nwgt--\>/g,">").replace(/\<[\/]?tbody\>/gi,"");

(Why they hate the poor TBODY so much they must strip it from the markup even though the browser will re-generate them in the DOM as soon as innerHTML is parsed I can't even begin to imagine.)

If you thought XML-based toolchains and processes were going to make the Web a saner place, think again. We have now seen that in the right hands, XSLT is just another recipe for tag soup.

A colleague just noticed that the otherwise excellent jQuery JS library has an "isFunction" method that relies on decompiling functions to figure out if they really, really are functions. This is not a good idea because decompiling functions is an optional feature of the ES-262 spec. It is slow and requires too many resources for certain low-end platforms, so it is not supported by any of the Opera Mobile versions.

Here's the code, complete with a frustrated comment from John Resig himself:

// This may seem like some crazy code, but trust me when I say that this
// is the only cross-browser way to do this. --John
isFunction: function( fn ) {
return !!fn && typeof fn != "string" && !fn.nodeName && 
fn.constructor != Array && /function/i.test( fn + "" );
}

John, I feel your pain and that code clearly took quite some effort and testing across various browsers passing in various types of objects - IE collections allowing ()-reference, anyone? It wouldn't surprise me if Opera is one of the culprits that caused you problems here since we try to support some of IE's weirdness and hence "typeof" sometimes returns 'function' when you'd expect it not to.

So I'm sorry to break it to you, but /function/i.test( fn + "" ) won't do what you think on Opera's mobile versions, causing really-hard-to-track-down bugs.

May I suggest adding the below workaround somewhere?

if( (function(){}).toString().match(/\^[ecmascript/i) ) Function.prototype.toString = function(){return 'function'};

Should be all you need..

If you ever came across code that starts

eval(function(p,a,c,k,e,r)

here's a small demonstration of how to turn that code into something sensible..

A word of warning: I'm experimenting with Wink for creating Flash screencast-type demonstrations. That means you agree to receive a 1.5 M .swf file of screenshots and text by clicking through to the post.

I haven't quite mastered Wink, some unintentional mouse movement might occur but anything in the spirit of experimentation.. Feel free to tell me in comments whether you like more of those Flash-based visual demonstrations or prefer the old-fashioned screenshot approach.

Read more...

The DOMStorage feature of the WHATWG WebApps spec is criticised for the security implications. 0x000000 has excellent points (particularly in comments) and I'll think out loud about how UAs like Firefox and Opera will need to address them. I'm not terribly good at spec writing and not implementing this stuff myself so only personal opinion and thoughts to be found here..

Quoting 0x000000:

Starting in Firefox 2 but mainly in Firefox 3, the browser is fully capable of restoring this sessiondata after a crash. This is interesting because we could on purposely crash a window and inject sessiondata to make it an persistent denial of service

Yes, that's a real threat. Opera already addresses a similar issue in the dialog you get on resuming after a crash, where you can choose to start without open windows. An even better implementation might be to detect what window or session caused a crash, and attempt to restore all others - I assume it would require quite some engineering like a separate monitoring process or something. In any case, users will need a "start without restoring sessions" option to avoid persistent session storage enabling persistent crashers.

this storage engine allows me to store what ever I want, whenever I want, and it allows huge data -if I have access to that page- which is useful when there is some XSS hole on that page

Um, if there is an XSS hole on the page you've got the data anyway, is this really more dangerous than just submitting it to a server you control?

It can be used to store worms and activate them across webpages instead of calling remote scripts

Still requires some XSS vulnerability on the page you want to run the worm on. If white-list restriction is implemented it requires both attacker and target to be white-listed.

it can used to store heapspray shellcode to actively stay under the radar of anti virus programs

The bad code needs to travel over the wire at least once to get to the local storage, so that's not entirely true but still an interesting point. I guess it's a matter of timing. If you can get a piece of malware into the local storage of a sufficient number of UAs before security software is updated to detect this threat, you could then activate the attach with some fairly innocent-looking JS that might be much harder to filter out. Should UA vendors collaborate with Antivirus companies on enabling local storage scanning? I'm not sure here, guess malware detection evasion side effects of local storage need more thinking..

it can store a great deal of information later to used on sub domains to read them out again and use them, like defeating single sign on schemes

I'm not sure what he means by "defeating single sign on schemes" here.

it can also be used to profile users and de-annonymize them by storing data vectors in their browser

Yes, it can if you find an XSS hole on a site I give personal information to (or if a web developer by mistake stores given information in a too public part of the local storage).

Now, security concerns are addressed by the specification. Perhaps it's too easy to overlook, but note how the spec explicitly states that

user agents may prevent domains from storing data in and reading data from the top-level domain entries in the globalStorage object

and I guess UAs will in practice require per-domain white-listing of domains that are allowed to use global storage, or make it available only in "application-like" contexts such as widgets. I can't imagine any UAs implementing this part of the spec without additional security restrictions, that would indeed be crazy. On the other hand, per-domain whitelisting will address most of his potential exploits as long as the white-listed sites themselves are safe.

Nevertheless, the spec gives greater powers to web developers. It will enable them to do interesting things more easily, but as we know with greater powers comes greater responsibility - including responsibility for coding responsibly and preventing XSS. The greater capabilities we give JavaScript, the greater the damage a single XSS exploit can cause.

(Opera doesn't support DOMStorage yet but we probably will at some point, apparently Firefox 3 will support it according to 0x000000's post.)