Opera Core Concerns

Official blog for Core developers at Opera

Persistent client side storage for your persistent needs

In the beginning of 2009 we started developing support for the persistent storage APIs, Web Storage and Web SQL Databases. Meanwhile, with the Opera 10.50 release cycle, the feature was made public.

Despite being two separate APIs, they share some common design. Both benefit from quota control, which can be tweaked in opera:config. Whenever a web site tries to go over its allowed quota, the user will be requested for more. Privacy mode, delete private data and widgets are also obviously supported. All web storage areas and databases are visible in opera:webstorage and opera:webdatabases respectively. Lastly, all data is placed under the folder "pstorage" in your profile folder.

Currently, we support all features in the Web Storage specification, with the exception of structured storage and the storage mutex. Testing also showed that our Web Storage implementation has nice performance.

Naturally, we chose SQLite as the database engine for Web Databases. SQLite, being in the public domain, allowed us to include it in Opera. Also, we can ensure compatibility with the other browsers that also support Web Databases. SQLite integration went without issues: the online documentation is very good, and allowed us to do everything we wanted. We commend the SQLite project for the great work they are doing.

Unfortunately, the binary footprint of SQLite does not make it perfectly usable in many of the less powerful devices that Opera ships on, so the feature might not be available everywhere. Therefore you should always do proper feature detection before attempting to use the feature, as shown below.
if (window.openDatabase)
{
    /* use web databases */
}
else
{
    /* fallback */
}

Some people have asked if we should support replacing the database engine with something else, like MySql. Currently, there are no plans to do so. Other databases engines are not perfectly interoperable with SQLite, and the integration with an external DBMS is not trivial.

Web Storage support in Dragonfly is already available, and we hope in the future to have nice SQL development tools builtin. For details about the APIs you can refer to the specifications linked above, and to dev.opera.com.

Everything you need to know about HTML5 video and audioWebSockets in Opera

Comments

Charles SchlossChas4 Wednesday, March 3, 2010 3:15:53 PM

up

Barraco Mármol Jerónimojerobarraco Wednesday, March 3, 2010 3:33:10 PM

i just hope that webpages dont bloat us with databases...
this is very weird feature when you analize it.
there was a trend actually to remove all information on the client side this would be like reverting that model.

i can't really imagine how this could be used in a useful, intresting and justifyng way.

Abhinavdecodedthought Wednesday, March 3, 2010 3:40:00 PM

up

Martin KadlecBS-Harou Wednesday, March 3, 2010 4:02:42 PM

Originally posted by jerobarraco:

i can't really imagine how this could be used in a useful, intresting and justifyng way.



For Unite Apps and Widgets It's amazing technology and gives us awesome possibilities.

Barraco Mármol Jerónimojerobarraco Wednesday, March 3, 2010 4:09:18 PM

Originally posted by BS-Harou:

For Unite Apps and Widgets It's amazing technology and gives us awesome possibilities.


oh, you're right, i totally forgot about them, i was only thinking about webpages..

dahulevogyre Wednesday, March 3, 2010 4:25:18 PM

Isn't Apache Derby more portable ?
It only requires JVM 1.4 or higher.
It's distributed under Apache License 2.0, which is ok for opera i think.
It's light, and more powerful and robust than SQLite.

lucideer Wednesday, March 3, 2010 4:36:16 PM

Originally posted by jerobarraco:

i was only thinking about webpages..


While they haven't proven to be absolutely massively popular, there are still quite a few people interested in things like "Offline Gmail", and similar apps - they've mostly relied upon Google Gears, but as that's now defunct such things can move to using standardised persistent offline storage like these.

svdb0 Wednesday, March 3, 2010 5:40:53 PM

This is a step in the wrong direction, imho.

I would think that with the whole web being based on XML/SGML derived standards, using an XML database with the existing XPath web standard as a query language would be the obvious choice.

But apparently not. For some unexplained reason, it was decided that the way forward is to add the ill-fitting SQL Sqlite to the web standards.

(Another, easier to use way to permanently store data would be through persistent objects. I guess that now that we have getters and setters, this could — and probably will eventually — be implemented in a JavaScript library. But even then, there would be the need for a query language.)

lucideer Wednesday, March 3, 2010 6:55:21 PM

@svdb0
I must say I agree with everything you've said. Non-SQL DBs are fast coming into the mainstream (CouchDB anyone?) - this seems like a bit of a shortsighted move (certainly not the only one in the HTML5 spec either). However, taking issue with it now is also a bit late given the first draft was almost a year ago.

FransFrenzie Wednesday, March 3, 2010 8:15:36 PM

Originally posted by svdb0:

I would think that with the whole web being based on XML/SGML derived standards, using an XML database with the existing XPath web standard as a query language would be the obvious choice.

But apparently not. For some unexplained reason, it was decided that the way forward is to add the ill-fitting SQL Sqlite to the web standards.

XPath is insufficient on its own. XQuery would be fine with me from what I read though (and it essentially supports/includes XPath). In the end I guess it's that millions (including me) already know SQL and "we" don't know XQuery.

Haavardhaavard Thursday, March 4, 2010 7:43:29 AM

Originally posted by jerobarraco:

i can't really imagine how this could be used in a useful, intresting and justifyng way.

Caching data so that you can keep working even if your connection drops (for example when on the train, going through a tunnel)? I would actually find that very useful.

Keep in mind that more and more people browse on the move. Wireless connections aren't necessarily as stable or reliable as the plug in the wall.

sirnh1 Thursday, March 4, 2010 8:51:41 AM

Would it be possible to get an extra tab in the site preferences, with the ability to view all data of that particular site (just like with cookies, but then for client storage)?
(maybe it would also be nice to show the 'allowed quota' for the current site, with the ability to change it)

João EirasxErath Thursday, March 4, 2010 5:31:17 PM

Originally posted by sirnh1:

with the ability to view all data of that particular site (just like with cookies, but then for client storage)?


Dragonfly does it

Originally posted by sirnh1:

(maybe it would also be nice to show the 'allowed quota' for the current site, with the ability to change it)


and opera:webstorage and opera:webdatabases do it too smile

And you're free to touch the file on your hard drive. But be careful with what do you, else you might lose your data.

svdb0 Thursday, March 4, 2010 8:07:05 PM

Originally posted by lucideer:

However, taking issue with it now is also a bit late given the first draft was almost a year ago.


This isn't the first time that I've brought this up. About nine months ago I actually asked for the rationale behind the design on the official [a href="http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/1174.html"]WHATWG public mailing list[/a]. Noone replied.

While there have apparently be more doubts about this issue, the people who matter (the browser builders) seemed to have decided the issue (individually?) off the list.

I suppose that it's a fait accompli — and kudos to Opera for trying to move the web forward — but that doesn't change the notion that the wrong path has been taken. And so I consider it appropriate to speak out to put this development into perspective.

Cutting Spoonhellspork Friday, March 5, 2010 4:44:04 AM

Great concept even now, only time (and committee) will determine the way forward. I personally feel that many of these specs are not sufficiently objective, neither are they considerate of the new push for small/mobile devices.

sirnh1 Friday, March 5, 2010 8:46:45 AM

Originally posted by xErath:

Dragonfly does it


Originally posted by xErath:

and opera:webstorage and opera:webdatabases do it too :smile:


I know But I thought that it would be nice to have all of that in one place (so I wouldn't we wouldn't have to go all over the place to change the settings of a certain site.
(I don't really know how the opera:webstorage and opera:webdatabases would look like since none of the sites I regularly visit use it. I already did notice that those 2 still have the old 'opera 9' looks (unlike the opera:about page that has the more modern look))

Leonardo Alassialeoalassia Wednesday, April 7, 2010 8:04:36 PM

I've made some examples with sql, but I get confused...
First I've programmed a code to create a table, insert rows and show the selected rows... everything ok... but then I've changed my code to just select the rows and reload the page... it seems like everything was removed, the table hasn't any records, it's suppossed to work like that?

Jan Lundquistjolo1701 Thursday, May 27, 2010 11:30:42 AM

I've written a web-app that uses this, but just now found out that foreign key constraints doesn't work. I can define tables with the constraints, but it's not enforced. As of now i haven't found at way to enable it :-(

João EirasxErath Monday, October 11, 2010 9:22:43 AM

Originally posted by Jan Lundquist:

but just now found out that foreign key constraints doesn't work


No, SQLite does not support that. You could try triggers ?

Write a comment

New comments have been disabled for this post.