A while ago I was asked to do quality assurance on our upcoming
JSON implementation. We support the JSON grammar and API specified in ECMAScript edition 5,
ES5. ES5 was still being edited and changed while we were implementing and testing, but with the patient help of members of the ES-discuss list I got a
test suite written and aligned with the upcoming spec. Stanislav - our JSON-implementor - had to be patient with me as tests were continually changing their pass conditions. For example, one day we were testing for the presence of a certain method, then the next day its absence!
At this point our implementation is considered ready for release, and I'm pretty excited about it - it was a pleasure to test, and Opera's native JSON support is going to be fast, strict, safe and standards-compliant. According to our tests our implementation is the fastest one at parsing -
| Simple parse |
| Opera |
150 000 times/second |
| Safari |
127 000 times/second |
| IE |
41 000 times/second |
| Firefox |
37 500 times/second |
though we're a bit slower than the others at serializing:
| Simple stringify |
| Safari |
71 000 times/second |
| Firefox |
61 800 times/second |
| IE |
57 000 times/second |
| Opera |
55 800 times/second |
..so there is a bit of work left on optimizing that. Keep in mind that this is with our existing JavaScript engine - the new
Carakan engine gives JSON parsing a solid boost too!
Also, as you would expect from Opera, our JSON support is following the standard very closely. We are pretty standards-compliant, better than IE8 and Firefox 3.5, and on par with Safari 4.
Actually, make that
almost standards-compliant. There are a couple of extremely odd edge case bugs that most likely won't cause you any problems - and there is
one spec violation that we've decided to keep. Blame me for that decision, since I asked Stanislav to break his perfectly compliant implementation and ignore the specification here. Sincere apologies to
Douglas "Mr. JSON" Crockford...
Why violate the specification?
Let's take a step back: JSON has become hugely popular on the Web. It's a simplistic and flexible format for data exchange, parses quickly and integrates well with programming languages. It looks like (and is named after) a JavaScript object literal, so it's not surprising that JavaScript authors make good use of it, traditionally parsing it with
eval(), loading it directly into a
SCRIPT element, or using a custom parser written in JavaScript.
For certain use cases JSON is replacing XML for data exchange, mainly since it can be parsed faster. However, the languages have something in common: both have
Draconian error handling.
Draconian error handling in JSON? Indeed, except it's nowhere near as controversial as XML's Draconianness, because JSON looks and feels like a programming language. We're all used to having to match our quotes and curly braces when programming. However, the spec grammar has a few additional constraints - a few points where correct JSON is different from and stricter than normal JavaScript source code, such as:
- No hexadecimal number formats are allowed - {"number":0x1} is wrong!
- Strings must be quoted with double quotes, not single - {"string": 'a string'} is wrong!
- TAB characters may not appear inside strings - {"tab":"there is a tab -> <- here"} is wrong! The spec requires using \t instead of a literal TAB character.
The point of these constraints is to keep the syntax and JSON language simpler. However, when browsers start adding native JSON support, our implementations will enforce those constraints - something parsing JSON with
eval() or
SCRIPT tags never did. Hence, we're going to deal with potential compatibility issues with invalid JSON.
The spec rule Opera is going to violate for now is the one about not allowing tab characters inside strings. Currently all other browsers allow it (and if your browser fails
this test case, it does too). I would like us to align with the spec, but I'm worried that we might break websites that do not escape tabs in their JSON data. If there are such "compatibility taxes" to be paid, I'd rather not have our users pay them.
As an example of what might break, I sent my GMail address an E-mail with a tab character in the subject field, and saw that GMail passed the tab around in its JSON data, unescaped. If GMail is using the browser's native JSON parsing and we follow the spec, somebody could then prevent your inbox from loading in Opera by sending you a message with tabs in the subject. And given the current browser implementations it would only break your GMail in Opera...
All browser developers discussing this on the es-discuss list have comitted to aligning strictly with JSON grammar - so
it should be only a matter of time before other major browsers do the right thing. When others pass
the test we'll certainly follow. Meanwhile, enjoy faster and safer web applications thanks to a quality JSON implementation soon shipping in an Opera browser near you.
Aux # Friday, December 18, 2009 12:02:35 PM
MyOpera team, please fix this!fearphage # Friday, December 18, 2009 12:17:57 PM
Rafael Luikrafaelluik # Friday, December 18, 2009 12:27:13 PM
Robert MeijersRobert90 # Friday, December 18, 2009 12:28:03 PM
Originally posted by Aux:
You can probably check this using if(JSON.Parse) (which is already supported by Safari and Firefox if memory serves me right)
Hallvord R. M. Steenhallvors # Friday, December 18, 2009 12:28:51 PM
Checking for native JSON is easy -
To be extra thorough you could also check that Date.prototype.toJSONString is a function.
Hallvord R. M. Steenhallvors # Friday, December 18, 2009 12:44:07 PM
Martin RauscherHades32 # Friday, December 18, 2009 1:18:48 PM
If the implementation is ready for release, will it be in 10.2 Final?
Aux # Friday, December 18, 2009 1:57:37 PM
zoquete # Friday, December 18, 2009 2:31:18 PM
...and if your browser fails this test case, it does too)
cousin333 # Friday, December 18, 2009 4:30:46 PM
1. test: IE has about the same speed as FF
2. test: IE is about 8-10% slower than FF
That nearly covers your findings.
MagicM # Friday, December 18, 2009 8:03:53 PM
RFC 4627 says that the MIME type is application/json and that the character set is UTF-8, UTF-16, or UTF-32.
What if the wrong MIME type is sent, such as text/plain?
What if a MIME type of text/plain is sent with a character set of iso-8859-1?
Hallvord R. M. Steenhallvors # Friday, December 18, 2009 11:13:25 PM
cousin333: I put those numbers together last week or so, as far as I know latest final versions of all tested browsers except Firefox, since I was not aware that it had silently failed to update itself. The testing was fairly unscientific on a computer with various processes running but numbers were fairly consistent across reloads and seemed good enough to quote. Glad your results are comparable
MagicM: MIME type is not really relevant, typical use case is a script loading data through XMLHttpRequest and in such cases we just pass any textual data returned from the server to the script regardless of HTTP MIME type. If encoding is mislabelled I assume it will depend on your luck and what characters the text contains exactly - something outside US-ASCII mislabelled as iso-8859-1 will likely turn into characters that makes the source text invalid JSON data and parsing will throw an exception. I guess a few tests for this would be useful.
Kirilljarinkirill # Saturday, December 19, 2009 2:10:35 PM
porneL # Saturday, December 19, 2009 4:12:12 PM
David Storeydstorey # Saturday, December 19, 2009 4:42:17 PM
Charles SchlossChas4 # Saturday, December 19, 2009 7:28:32 PM
Barraco Mármol Jerónimojerobarraco # Saturday, December 19, 2009 11:01:28 PM
Hallvord R. M. Steenhallvors # Saturday, December 19, 2009 11:28:22 PM
David: you can simply use the "Quick Find" field at the top right of the runner - type in "FAILED" to see only failures. Or use any other filter string, of course
Martin RauscherHades32 # Tuesday, December 22, 2009 10:51:26 AM
Simple parse 174000 times in 1002ms
Simple stringify 140000 times in 1004ms
WOW! Opera CRUSHED them all!
Charles SchlossChas4 # Tuesday, December 22, 2009 3:56:03 PM
svdb0 # Thursday, December 24, 2009 4:57:12 PM
Originally posted by hallvors:
Whoa, that's quite a turnaround.
Is this an exception, or does this signify a shift in the "Open the Web" principles?
Purdi # Thursday, December 24, 2009 7:37:53 PM
Originally posted by svdb0:
What are you talking about? There's no turnaround at all here. Opera has always tried to support all sorts of non-standard code to work with real sites.
It's a myth and disgusting lie that Opera is so damn strict and only accept standards compliant code.
Hallvord R. M. Steenhallvors # Friday, December 25, 2009 12:10:40 AM
Originally posted by svdb0:
I wouldn't go as far as calling it a "turnaround". For many years we've been trying to follow the standards, yet often had to make amendments where other browsers were incompatible - due to bugs or different readings of some specification. Most of the time we only did so after end users had been experiencing and reporting problems due to the incompatibility. In this case I'm simply trying to apply our experience from those issues to our implementation and foresee what problems might occur.
Validating JSON input to make sure authors have changed TAB characters to \t is a minor feature for a few developers. Loading and rendering the GMail inbox is a very important feature for millions of users...
I'll note that our JSON implementation already is stricter than others for some aspects of parser validation (for example, when I was testing IE8 it allowed hex numbers. We don't. We also at times follow the spec more closely than the Firefox version I was comparing against - I don't off the top of my head remember what tests exactly and the latest Firefox may already have fixed things).
Opening and improving the web still matters. Spending hours on developing a test suite and aligning it with the spec is sufficient proof that spec-compliance remains important at Opera.
svdb0 # Sunday, December 27, 2009 7:42:59 PM
Also, I do not know Opera as people who hold of doing "the correct thing" until other browsers do it.
I'm not disputing that this may be the right decision and (unlike many other standard violations) accepting tab characters in JSON strings does not seem to do any harm.
My surprise came from the observation that this approach seems to be more pragmatic and less idealistic than what I'm used to from Opera. Which I guess you explained by learning from experience to anticipate problems.
But I am glad to hear that Opera is still (by and large) committed to standards. I wholly support that.
Edit: I do apologize if I sound harsh. Native JSON support in Opera is obviously a big step forward, even if it isn't perfect yet.
Charles SchlossChas4 # Sunday, December 27, 2009 7:54:19 PM
Originally posted by svdb0:
That is something a user may make to modify a site not Opera
All browsers support web standards by default so no it is up to the sites to fix their code to follow the web standards
I ran this
http://testsuites.opera.com/JSON/performance/001.html
Results:
Got score of 4.2405063291139244. The input was parsed 237000 times in 1005ms
On wine via Crossover on my mac with 3gb of drr2 ram and dual core 2.33 ghz cpu, Opera is fast
svdb0 # Sunday, December 27, 2009 11:02:38 PM
Originally posted by Chas4:
Sorry, I meant Browser JavaScript (browser.js).
Purdi # Monday, December 28, 2009 1:59:45 AM
Originally posted by svdb0:
How do you explain stuff like document.all then?
svdb0 # Monday, December 28, 2009 1:39:18 PM
Originally posted by Purdi:
I don't know what you are referring too. Could you give me a bit of context?
Michael A. Puls IIburnout426 # Monday, December 28, 2009 2:03:18 PM
Originally posted by svdb0:
Opera added support for the non-standard document.all instead of just faking it with UserJS on sites that needed it.
But, document.all is kind of a de-facto standard now and HTML5 in its current state defines it.
Hallvord R. M. Steenhallvors # Monday, December 28, 2009 11:11:13 PM
Originally posted by svdb0:
Idealism on behalf of specs and idealism on behalf of users can conflict
MyOpera team, please fix this!fearphage # Friday, February 26, 2010 3:10:43 PM
Garrett Smithdhtmlkitchen # Tuesday, June 15, 2010 10:52:13 PM
That's not acceptable when implementations are buggy, and Firefox and IE currently have buggy implementations of JSON.parse. Do you really want to use a feature that correctly throws errors in Opera, but parses "1." in Firefox and IE?
An application that generates numbers as "1.11" often, but seldom generates an invalid "1.", the program bug (of providing an invalid "1.") would not be immediately obvious.
Moreover, to be standard, test should use global object instead of window:
And even at that, there's no telling what
JSON.parse("1.")will do.You also wrote:
The function is
Date.prototype.toJSON. Checking that it is function does not guarantee that the function is implemented correctly. We've just seen how JSON.parse is buggy in some implementations andDate.parse("1999-01-01")is buggy in Firefox. Example:new Date("1999-01-01").getFullYear();Firefox 3.6.4: 1998.
I left more feedback on es-discuss list regarding this test suite.
Garrett
Garrett Smithdhtmlkitchen # Tuesday, June 15, 2010 10:54:06 PM
Hallvord R. M. Steenhallvors # Wednesday, July 14, 2010 9:12:00 PM
JanGen # Thursday, November 4, 2010 2:49:14 PM
Actually Chromium passes the test
7.0.517.41 (62167) Ubuntu 10.10
Hallvord R. M. Steenhallvors # Tuesday, July 12, 2011 12:25:24 PM