Cross-domain XMLHttpRequest

Cross-domain XMLHttpRequest

I, here by, give you a implementation of the regular XMLHttp object that supports fetching documents in other domains.
This first implementation works the same as a regular XMLHttpRequest with the following differences:

• Requests are always asynchronous
• there's a ontimeout event handler, which fires when the request exceeds a timeout value in milliseconds specified in the timeout property of the object, since the remote server can be offline. Defaults to 10000 ms

The script protects itself from being used by a regular webpage. It can only be used by other user scripts.
Therefore the script is sub-divided in 2 modules: the stack trace module and the xmlhttp object module.

As you might guess it uses frames and cross domain messaging.

Get it here (these will be updated):
a-lib-stacktrace.js
a-lib-xmlhttp-cd.js

Also you may use this test page (it's zipped for the sake of the forums script ripper):
crossdomain-xmlhttp-testpage.zip

To test, place the script in your user js folder, open the page and paste this piece of script somewhere

addEventListener('load',function(){

  var b = document.getElementById('opera-xmlhttp-test');
  if( !b || !(b instanceof HTMLBodyElement) )
    return;

  var xmlhttp = new opera.XMLHttpRequest();

  xmlhttp.onreadystatechange = function(){
    var s = ''+this+'\n';
    s += 'status: '+this.status+'\n';
    s += 'statusText: '+this.statusText+'\n\n';
    s += 'allheaders: '+this.getAllResponseHeaders()+'\n\n';
    s += 'responseText: '+this.responseText+'\n\n';
    s += 'responseXML: '+xml(this.responseXML)+'\n\n';
    document.getElementById('t'+this.readyState).value = s;
  }
  xmlhttp.onload = function(){
    document.getElementById('fff').style.display = '';
  }
  
  xmlhttp.open('GET','http://www.opera.com/',true);
  xmlhttp.send('a=b&c=d');

},false);

If the xmlhttp object succeeds and fetches the page, you should see the several textareas in the test page getting filled with data.
The 5th textarea, contains all data that the page can trap using a message event listener.

Now I ask you to try to crack it as hard as you can, so I can make a bullet proof script for later use.
If you can find a way to override native methods in Opera for the script security to fail, please report it.
I've already placed several hacks in the testpage, and fixed the script to work around them.

Have fun

I have the same problem as Fremean. I've tried the latest nightly build but with no success. Can it be fixed?

Originally posted by GoblinHero:

I have the same problem as Fremean. I've tried the latest nightly build but with no success. Can it be fixed?

I said it can't. It's an Opera bug that completely blocks cross-frame messaging, on which the script heavily depends and there's no workaround.

Originally posted by xErath:

It's an Opera bug that completely blocks cross-frame messaging, on which the script heavily depends and there's no workaround

I am using cross document messaging right now. how is it broken?

Originally posted by fearphage:

I am using cross document messaging right now. how is it broken?

window.parent throws.

function handleMessage(e) {
  parent = e.source;
  postData(e.data); // reply
}

function postData(request) {
  parent.postMessage(/* response here */, '*');
}

if (window.addEventListener)
  window.addEventListener('message', handleMessage, false);
else if (window.attachEvent)
  window.attachEvent('message', function() { handleMessage(window.event) });

works like a charm.

xErath: Maybe it throws due to missing second parameter in postMessage call?

eh ! Found the real problem.

Opera was throwing a security exception in the line postMessage.call(window.parent,...) on the child frame because the postMessage call invoked the acknowledge listener in the parent frame, which in turn sent a new message to the child with the XHR data to request. The child then XHR'ed google.com which was redirected to google.pt here, so when calling send() the security exception crawled all the way back the the 1st postMessage, hence my confusion.
I changed the test url to google.pt and then Opera did not dispatch onreadystatechange, because all the postMessage calls are synchronous, which is against the spec (which changed recently, before was synchronous).

I uploaded a new version with a workaround.

I've got two errors with test page, the first (t1) and the last (crap) textarea are empty.

JavaScript -
Unknown thread
Error:
name: Error
message: INVALID_STATE_ERR
stacktrace: ...

JavaScript - http://www.opera.com/
User Javascript thread
Error:
name: TypeError
message: Statement on line 21: Cannot convert undefined or null to Object
Backtrace:
Line 21 of User JS script
document.getElementById('crap').value = '';
stacktrace: ...

I've used this "caller" :

addEventListener('load',function(){

  var b = document.getElementById('opera-xmlhttp-test');
  if( !b || !(b instanceof HTMLBodyElement) )
    return;

  function xml(node){
      return (node && node.nodeType) ? new XMLSerializer().serializeToString(node):'('+node+')';
  }
  var xmlhttp = new opera.XMLHttpRequest();

  xmlhttp.onreadystatechange = function(){
    var s = ''+this+'\n';
    s += 'status: '+this.status+'\n';
    s += 'statusText: '+this.statusText+'\n\n';
    s += 'allheaders: '+this.getAllResponseHeaders()+'\n\n';
    s += 'responseText: '+this.responseText+'\n\n';
    s += 'responseXML: '+(xml(this.responseXML))+'\n\n';
    document.getElementById('t'+this.readyState).value = s;
  }
  xmlhttp.onload = function(){
    document.getElementById('fff').style.display = '';
  }
  
  xmlhttp.open('GET','http://www.opera.com/',true);
  xmlhttp.send('a=b&c=d');

},false);

I've made something wrong or there is another problem?

The javascript error is a bug in the test page. The t1 testarea should have data, I need to check. The crap textarea must be empty, else the script is leaking data.

I just did a whole bunch of fixes, and those problems now should be gone.
Now onreadystatechange is properly dispatched for both cross-domain and samedomain requests.

I can't load "Reader" since 2 days but if I remove the script of folder, It's working OK
Is it the problem of script with any change of greader?

Excuse my English

Originally posted by Shin_ta:

I can't load "Reader"

reader ??

Originally posted by xErath:

reader ??

Originally posted by Shin_ta:

greader

I guess, "google reader"...

I'm getting "message: INVALID_STATE_ERR: XHR::getAllResponseHeaders()" with the current Opera 10 Snapshot...

And when I remove the problematic lines I only get ONE event, for the state "0"...

Originally posted by Shin_ta:

I can't load "Reader" since 2 days but if I remove the script of folder, It's working OK

I did a quick try of google reader and it worked fine.

Originally posted by Hades32:

I'm getting "message: INVALID_STATE_ERR: XHR::getAllResponseHeaders()" with the current Opera 10 Snapshot...

Script is working fine. getAllResponseHeaders() can only be called when readyState>=3.

Hello !

I'm having troubles with your scripts... and didn't find any solution through the thread.

Here it is : when I try to load the test page, Opera's Error Console says that :

JavaScript - file://localhost/C:/Program%20Files/Opera/scripts/crossdomain-xmlhttp.html
Event thread: load
Error:
name: Error
message: Security violation - user scripts only
stacktrace:   Line 182 of User JS script : In function checkScriptRights
    			throw new Error("Security violation - user scripts only");
  Line 326 of User JS script : In function XMLHttpRequestCD
    			checkScriptRights();
  Line 72 of inline#1 script in file://localhost/C:/Program%20Files/Opera/scripts/crossdomain-xmlhttp.html
    		var xmlhttp = new opera.XMLHttpRequest();
  ...

I did put this .html file in the user js folder (it's /Opera/scripts/), which is the one I chose in Opera's settings !

It looks like the problem comes from this part of the test page :

addEventListener('load',function(){

  var b = document.getElementById('opera-xmlhttp-test');
  if( !b || !(b instanceof HTMLBodyElement) )
    return;

  var xmlhttp = new opera.XMLHttpRequest();

  xmlhttp.onreadystatechange = function(){
    var s = ''+this+'\n';
    s += 'status: '+this.status+'\n';
    s += 'statusText: '+this.statusText+'\n\n';
    s += 'allheaders: '+this.getAllResponseHeaders()+'\n\n';
    s += 'responseText: '+this.responseText+'\n\n';
    s += 'responseXML: '+xml(this.responseXML)+'\n\n';
    document.getElementById('t'+this.readyState).value = s;
  }
  xmlhttp.onload = function(){
    document.getElementById('fff').style.display = '';
  }
  
  xmlhttp.open('GET','http://www.opera.com/',true);
  xmlhttp.send('a=b&c=d');

},false);

too bad... what's wrong with what I did ? :/


When I remove the line #182 of the .js script (verification oh the rights of the script), it seems to work : the two first textareas are filled, but then it stops and the consoles tells me that

JavaScript - file://localhost/C:/Program%20Files/Opera/scripts/crossdomain-xmlhttp.html
Event thread: BeforeEvent.message
Unhandled exception: "Security violation"

:/

what do I have to do to fix this ?


Thank you very much !

Originally posted by brunocauet:

message: Security violation - user scripts only

Only user scripts can use that object.

Originally posted by brunocauet:

: the two first textareas are filled, but then it stops and the consoles tells me that

working fine here. which url are you trying to fetch ?

Originally posted by xErath:

Only user scripts can use that object.

Is it a bug that userjs can override built-in security models?

Originally posted by fearphage:

Is it a bug that userjs can override built-in security models?

It uses postMessage which is ment to be used cross-domain.

Originally posted by d.i.z.:

It uses postMessage which is ment to be used cross-domain.

Haven't looked at the code in ages. Indeed my mistake. It used to override Opera's security.

xErath, I'm trying to open this page :
which is the one you posted in a zipped file in your first post.

Originally posted by brunocauet:

xErath, I'm trying to open this page :

You're trying to XHR that page ? hum... I haven't tried xhr local stuff from local files.

Hello.

I was trying to marry this with Opera XML Store to store preferences for some userjs scripts of mine and it went fine but I found one issue... the responseXML property of the response was always empty... after some extensive debugging I found that this line of code from the parseDOM function is the culprit:

var nd = DOMParser_parseFromString.call(new DOMParser(),
				(String_match.call(markup,/<!\s*doctype\b/i)?'':
				'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">')
				+markup,'text/xml');

It adds the doctype element if there is none in the response but it fails with the response from Opera XML store... it causes a parse error in the DOM parser...
Removing this part:

(String_match.call(markup,/<!\s*doctype\b/i)?'':
				'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">')
				+

and leaving just the mark alone as a second argument to the call fixes it...

Originally posted by nowotny:

I was trying to marry this with Opera XML Store to store preferences for some userjs scripts

That's a really nice idea !

Originally posted by nowotny:

the responseXML property of the response was always empty

The parseDOM function tries to parse the code as valie xml, but has a fallback to parse it as html. Is that branch being run ?
The doctype must be there, so entities like   can be recognized.

You have a link to the document that fails, or can save it somewhere ?

Originally posted by xErath:

The parseDOM function tries to parse the code as valie xml, but has a fallback to parse it as html. Is that branch being run ?

No, it's not...

Originally posted by xErath:

The doctype must be there, so entities like   can be recognized.

Oh... ok... I didn't know that...

Originally posted by xErath:

You have a link to the document that fails, or can save it somewhere ?

I'm attaching a working example of a userjs script... All you have to do is to put it in the userjs directory... before you run it you might want to change the @include parameter though... also, you need to have Opera XML Store account and be logged in and replace my username with yours at the very bottom of the script...

I left some notes in comments in your parseDOM function...

I'm also available on IRC if you want to contact me directly...
test.zip

Hello...

I think I found another small issue though I'm not really sure where lies the exact problem...
I set a custom header with setRequestHeader() and when doing POST requests it goes out fine but with GET it's not being send... I checked what's exactly send with Proxomitron... Could you check what's going on...?

I'm trying to use this library to connect to any other page than "http://opera.com" and i get... nothing?

I don't know what i'm doing wrong - i even can't get google page...

 var xmlhttp = new opera.XMLHttpRequest();

	xmlhttp.onreadystatechange = function() {
		alert(this.readyState);
	}
 
  xmlhttp.onload = function(){
    alert(this.responseText);
  }
  
  xmlhttp.open('GET','http://google.com/',true);
  xmlhttp.send();

Originally posted by Lashus:

xmlhttp.open('GET','http://google.com/%27,true);

google usually redirects to other tlds, like google.pl in your case. If so, the script will break, and I haven't had the time or patience to fix it.

I'm ok with that I don't want to use google page anyway ^^. However my page (http://lashus.one.pl/plemiona/wioski.php) is also not loading... it's nothing on it (i think) which could block script... Any idea what's wrong?

Btw, how could i debug it for checking what's wrong?

It works for me in Opera 10 (latest build).

Be sure that you run your script after Cross-domain XHR script runs. And look for any errors in message console.

In my script it's stopping on state 1... I haven't seen any errors in built-in "Error console" in Opera. I've gote Opera 10 (build 1551)...

window.addEventListener('load', function(e) {

 var xmlhttp = new opera.XMLHttpRequest();

	xmlhttp.onreadystatechange = function() {
		alert(this.readyState);
	}
 
  xmlhttp.onload = function(){
    alert(this.responseText);
  }
  
  xmlhttp.open('GET','http://lashus.one.pl/plemiona/wioski.php',true);
  xmlhttp.send();

}, false);

It works for me. Goes up to readyState 4 and fetches page source properly.

Paste your code please, let me see it - maybe i'll find mistake in that way...

@Lashus: I have used the code you posted. I've put that into xhr.js and refreshed this thread. It fetched content from your site without problems.

@xErath: any ideas regarding my posts...?
http://my.opera.com/community/forums/findpost.pl?id=2982505
http://my.opera.com/community/forums/findpost.pl?id=2985065

Originally posted by d.i.z.:

@Lashus: I have used the code you posted. I've put that into xhr.js and refreshed this thread. It fetched content from your site without problems.

Lol... then i really don't know what's wrong... thanks anyway

// EDIT:

Problem solved thanks to d.i.z.. It was problem with my browser ^^, great job with script btw

Originally posted by nowotny:

@xErath: any ideas regarding my posts...?

1)
If Opera sends the header for one request method but not for the other, then it can be an Opera bug most likely. Can you make a small testcase using a normal XHR ? btw, the url that is passed to XHR.open is first loaded using an iframe, and another XHR is done in that iframe requesting the same url but with the new headers. Can you observe that?

2)
Haven't looked at the xml store thignie yet...

Originally posted by xErath:

1)
If Opera sends the header for one request method but not for the other, then it can be an Opera bug most likely. Can you make a small testcase using a normal XHR ? btw, the url that is passed to XHR.open is first loaded using an iframe, and another XHR is done in that iframe requesting the same url but with the new headers. Can you observe that?

Ok... I checked the normal XHR and the headers are sent fine... this is not an Opera bug...

I tested again and I must say the randomness of this issue is mind boggling... :/ I mean, once it works then it doesn't work and I just cannot pinpoint the common denominator... It appears that with some urls it works properly and with some other it doesn't... I tested with a page on my own server and with XML Opera store page... see the attached file for the script I used to test this...

I noticed that it always works fine when I test it with a page on my server... http://nowotny.x10hosting.com/xhr/index.php returns just the contents of the $_SERVER array:

<?php print_r($_SERVER); ?>

and in that array I can always see that the header is sent properly...
I also opened the Dragonfly and observed the requests and responses Opera makes and I noticed that it properly makes 2 requests: first for the iframe and second from inside the iframe like you said... this second request sends my header and it's all good...

Then I tested with the XML Opera Store page and... well... results vary... :/ sometimes it worked, but most times it didn't... :/ I noticed it's because the second request was never sent... and here lies the real problem... the second request is not sent and I don't know why... :/ maybe you have some idea cause I'm out... :?

operastoretest.zip

Maybe it's the same issue that Lashus had. In some circumstances userjs can get disabled for some pages if you used Site Preferences in them.

Find profile/override.ini file and look for User Prefs|User JavaScript=0 line in specific sites that don't work.

Originally posted by d.i.z.:

Maybe it's the same issue that Lashus had. In some circumstances userjs can get disabled for some pages if you used Site Preferences in them.

Find profile/override.ini file and look for User Prefs|User JavaScript=0 line in specific sites that don't work.

No... UserJS does work... just incorrectly...

Also, I don't have any specific settings for this sites... they're not in override.ini...

Gah, it turns out in certain circumstances (HEAD requests on a URL that would normally generate a download dialog) this still cannot beat a native implementation.

Okay, so the page gets cached, and in BtEOs case, it generates a download dialog.
I would fix this by pointing the iframe to the root of the domain and not the url itself. It could have side effects, it could not. Depends on the website.

Yeah, I'd already though of that. It doesn't work with the domain in question, they redirect to a separate domain on an invalid URL, which the root is in this case.

Bah... I give up... :/

OSA, we need HTML5 local storage...

How about a Unite service that takes a GET request (via a script src), makes a cross-domain xhr on the server-side, packages the response and returns it as a script...

I've not written a Unite service as of yet, so I haven't gotten my head around the server-side javascript being used, but it sounds possible in my head.

+1 nowotny.

And +1 lucideer too, I've though the same thing Saturday when I try Unite.
But it seems to me to be too many heavy for that, it opens two sockets whereas only one should suffice if no hack was needed... :/

Object.prototype.__defineSetter__(
        "UserJsOnly",
        function(arg) {
        });
Object.prototype.__defineGetter__(
        "UserJsOnly",
        function() {
            return function() { return true; }
        });

What do I win?

Guys, Is there a way to make this http://userscripts.org/scripts/review/43451 work using this script ?

Originally posted by dapxin:

Guys, Is there a way to make this http://userscripts.org/scripts/review/43451 work using this script ?

Should work fine... what's the problem...?

It doesnt work for me. I tested it in FF <the only reason I ever load the junkware>

You tested what exactly because it will not work without a rewrite...

Originally posted by nowotny:

You tested what exactly because it will not work without a rewrite...

I tried this script http://userscripts.org/scripts/review/43451. It didnt work.

So I looked into the userjs and noticed it may need this script to work but I am not sure how exactly to let it work.

All I need to do is get this script http://userscripts.org/scripts/review/43451 to work.