Opera Dragonfly

Scope Protocol release: how the fat lady sings

By Johannes Hoff. Thursday, 2. July 2009, 13:50:23

Today we are happy to release the specification for the Scope protocol. This is the protocol used for communication between the Opera browser and Opera Dragonfly. It is also used here at Opera for automated testing.

Since the release of Opera Dragonfly, we have tried to keep the project open source. The source and its documentation is available under BSD licence, but it is difficult to expand it or create something different without the protocol specification. This release wants to rectify that.

The documentation is more or less as we have used it internally to communicate inside the team, which means that some documentation is missing simply because everyone on the team knew what it was about. If something is unclear, let us know in the comments.

One of the reasons we are releasing right now, is that other browser makers have started discussing how to do remote debugging. Since the very start we have focused specifically on this, so we want to share our experience and ideas with everyone else.

The specification is split into two parts. The current implementation, which we call Scope Transfer Protocol 0 (STP/0), is an XML based protocol. Our experience has been that this is too slow once you start transferring big data structures, like DOM trees. One of the advantages of the XML protocol was that it was easy to create a client for it in JavaScript using XMLHTTPRequests. It is also very easy to debug visually without the need for special tools to parse the data.

The next generation protocol, STP/1, is what we are currently working on. It is just a sign of things to come, but we will release it in a public build as soon as it is ready. There might be changes to the protocol before that time. The protocol continues to support XML, but we have added two more serializers: JSON and Protocol Buffers. The messages are the same in an abstract sense, but they can be rendered to different data structures. This means that we can keep our XML tools mostly unchanged, but switch over to Protocol Buffers for faster transmissions, or to JSON for the sweet spot between easy of use by JavaScript and low bandwidth usage.

Our next release will be a public build of Opera using STP/1, together with tools to help you get started with communication through STP/1.

Enjoy the read!

Opera Dragonfly alpha 4 snapshotImproved element highlight lands in Opera Mobile 10 Beta

Comments

MTKnight # 2. July 2009, 18:10

Fascinating. I'll be reading through it soon.

Chas4 # 2. July 2009, 19:18

jerobarraco # 3. July 2009, 14:35

really intresting

sigbjornfinne # 3. July 2009, 16:56

Cool,hope a common remote debugging protocol becomes a reality - is there a print-friendlier version of the spec
somwhere? (The .txt versions of stp/1 sort of work..)

cheers

aleto # 3. July 2009, 18:12

Originally posted by sigbjornfinne:

is there a print-friendlier version of the spec
somwhere?

There is a link "Show Source" in the side panel to the .txt version for each document.

sigbjornfinne # 3. July 2009, 22:29

thanks, that's what I've been using, but it's the raw markup per section/doc.

Specs beg themselves to be printed & annotated

aleto # 6. July 2009, 12:08

We have added a print style sheet, now it should be better.

sigbjornfinne # 6. July 2009, 15:15

awesome (looking); thanks

aleto # 7. July 2009, 13:09

You can now also get it as pdf.

sigbjornfinne # 13. July 2009, 10:37

This is good stuff, people interested in remote/cross-browser debugging/ers may also want to have a look at

http://groups.google.com/group/webdebugprotocol

sigbjornfinne # 13. July 2009, 10:42

minor wibble (quite literally

), the first example in the "Message flow" subsection of the STP1-DATA section uses '~' rather than '-' for the scope.Connect arrows in the first timing diagram. I believe it should be the latter (i.e., '-'.)

sigbjornfinne # 13. July 2009, 10:45

Inevitable question, I guess, but has there been any work on providing a CLI/console debugger (think: gdb and its remote serial protocol ) for Scope's EcmascriptDebugger service?

sigbjornfinne # 13. July 2009, 10:48

Apologies for the deluge

, but what's the release timeframe/vehicle for STP/1? Should downstream tool providers focus on STP/0 for now, and (easily) move along later?

dstorey # 13. July 2009, 18:37

I'd recommend focusing on STP/1 rather than STP/0. That is the version we are focusing on, and are moving to after Opera 10. It fixes a lot of issues that we've found and improved upon after experience of using STP/0 in the real world, such as speed issues.

We can't say with absolute certainty when the time frame for STP/1 is. It is a part of Presto 2.4, while Opera 10 is using Presto 2.2 (with patches). The next major version of Opera Desktop after Opera 10 will likely skip Presto 2.3 (which is already finished) and move to Presto 2.4. We'd like to get a Core Gogi build of Presto 2.4 out as soon as possible, so that it can be tested.

sigbjornfinne # 14. July 2009, 09:11

Thanks for release vehicle details, good stuff coming down the chute

Judging by the documentation, the base DOM API (i.e., the opera.scope* functions) won't change when STP/1 based versions are made available, so clients accessing Scope via it won't have to change all that much (modulo the extra params to some of the callbacks.)

Re: Scope services and their specification - is the syntax by which they are specified in Sec. 2.2 documented somewhere? It's mostly self-explanatory stuff; could it be rephrased in WebIDL form (at some point)?

FWIW, the Scope spec certainly has a slight edge on Bespin's debugger interface spec in terms of formality :-)

https://wiki.mozilla.org/images/8/80/Bespin_debugging_spec.pdf

May the two come together at some point,btw..

aleto # 14. July 2009, 09:32

About CLI: we have different tools in python, mainly for testing purpose ( e.g. one of them exposes the scope interface to http ). I think it wouldn't be too complicate to extend them, also it is not on our todo list ( but i hope we will be able to publish all related tools soon so that it would be easy to pickup from there ).

sigbjornfinne # 17. July 2009, 05:56

EcmascriptDebugger.ContinueThread and ThreadIDs:

is it ever the case that ThreadIDs aren't anchored to a runtime(ID)? i.e., would it make sense to hide their provenance in the API?

I see that ContinueThread()'s 'mode' field of has been clarified via a posting to the WebDebuggerProtocol group.

Kamerer # 24. July 2009, 18:17

Why did you stop sharing your cutting-edge releases at https://dragonfly.opera.com/app/cutting-edge/zips/ ?

d.i.z. # 24. July 2009, 21:07

Kamerer: they are here: https://dragonfly.opera.com/app/core-2-2/cutting-edge/zips/

I'm not sure but there might be some bug in redirection logic there.