Cross-domain XMLHttpRequest

Cross-domain XMLHttpRequest

I, here by, give you a implementation of the regular XMLHttp object that supports fetching documents in other domains.
This first implementation works the same as a regular XMLHttpRequest with the following differences:

• Requests are always asynchronous
• there's a ontimeout event handler, which fires when the request exceeds a timeout value in milliseconds specified in the timeout property of the object, since the remote server can be offline. Defaults to 10000 ms

The script protects itself from being used by a regular webpage. It can only be used by other user scripts.
Therefore the script is sub-divided in 2 modules: the stack trace module and the xmlhttp object module.

As you might guess it uses frames and cross domain messaging.

Get it here (these will be updated):
a-lib-stacktrace.js
a-lib-xmlhttp-cd.js

Also you may use this test page (it's zipped for the sake of the forums script ripper):
crossdomain-xmlhttp-testpage.zip

To test, place the script in your user js folder, open the page and paste this piece of script somewhere

addEventListener('load',function(){

  var b = document.getElementById('opera-xmlhttp-test');
  if( !b || !(b instanceof HTMLBodyElement) )
    return;

  var xmlhttp = new opera.XMLHttpRequest();

  xmlhttp.onreadystatechange = function(){
    var s = ''+this+'\n';
    s += 'status: '+this.status+'\n';
    s += 'statusText: '+this.statusText+'\n\n';
    s += 'allheaders: '+this.getAllResponseHeaders()+'\n\n';
    s += 'responseText: '+this.responseText+'\n\n';
    s += 'responseXML: '+xml(this.responseXML)+'\n\n';
    document.getElementById('t'+this.readyState).value = s;
  }
  xmlhttp.onload = function(){
    document.getElementById('fff').style.display = '';
  }
  
  xmlhttp.open('GET','http://www.opera.com/',true);
  xmlhttp.send('a=b&c=d');

},false);

If the xmlhttp object succeeds and fetches the page, you should see the several textareas in the test page getting filled with data.
The 5th textarea, contains all data that the page can trap using a message event listener.

Now I ask you to try to crack it as hard as you can, so I can make a bullet proof script for later use.
If you can find a way to override native methods in Opera for the script security to fail, please report it.
I've already placed several hacks in the testpage, and fixed the script to work around them.

Have fun

I did't see any result but a white pane.

Originally posted by jidixuelang:

I did't see any result but a white pane.

That's expected. The testpage should only work when the js sample above is in a user js file.

After more testing, I still couldn't find a way to break the script's security. Anyone want a go ?

And a fix to the test code !

addEventListener('load',function(){

  var b = document.getElementById('opera-xmlhttp-test');
  if( !b || !(b instanceof HTMLBodyElement) )
    return;

  function xml(node){
      return (node && node.nodeType) ? new XMLSerializer().serializeToString(node):'('+node+')';
  }
  var xmlhttp = new opera.XMLHttpRequest();

  xmlhttp.onreadystatechange = function(){
    var s = ''+this+'\n';
    s += 'status: '+this.status+'\n';
    s += 'statusText: '+this.statusText+'\n\n';
    s += 'allheaders: '+this.getAllResponseHeaders()+'\n\n';
    s += 'responseText: '+this.responseText+'\n\n';
    s += 'responseXML: '+(xml(this.responseXML))+'\n\n';
    document.getElementById('t'+this.readyState).value = s;
  }
  xmlhttp.onload = function(){
    document.getElementById('fff').style.display = '';
  }
  
  xmlhttp.open('GET','http://www.opera.com/',true);
  xmlhttp.send('a=b&c=d');

},false);

Nope, haven't managed to crack it. Just wanted to say, very nice idea, and good implementation, this is a keeper when considering the possibilities of userjs.

It's strange, but it doesn't work for me - nor in Opera 8.54, nor in latest 9.0 beta...

I broke it.

In the greasemonkey-functions.user.js, I redefined GM_xmlhttpRequest using your new opera.XMLHttpRequest object:

GM_xmlhttpRequest = function(options) {
  var request = new opera.XMLHttpRequest(), validEvents = { onload: null, onerror: null, onreadystatechange: null };
  
  function addXMLHttpRequestListener(request, eventName, callback) {
    request[eventName] = function(event) {
      var responseDetails = {
        responseText: request.responseText
        ,readyState: request.readyState
        ,responseHeaders: (request.readyState == 4 ? request.getAllResponseHeaders() : '')
        ,status: request.readyState == 4 ? request.status : 0
        ,statusText: request.readyState == 4 ? request.statusText : ''
      };
      callback.call(null, responseDetails);
    }
  }

  // add event listeners
  for (var eventName in validEvents) {
    if (options[eventName])
      addXMLHttpRequestListener(request, eventName, options[eventName]);
  }

  // open the connection
  request.open(options.method, options.url, true);

  // set the headers
  for (var header in options.headers) {
    request.setRequestHeader(header, options.headers[header]);
  }
  
  // send the data
  request.send(options.data);
  return request;
}

Then I entered this into the address bar:

javascript:void(GM_xmlhttpRequest({method: 'GET',url: 'http://virtuelvis.com/archives/2005/12/cross-document-messaging', onload: function(r) {alert(r.responseText)}}))

It showed me the source of the inputted url. I just did it again from my.opera.com.

aaa-greasemonkey-functions.user.js

Both of your scripts load before the greasemonkey-functions script.

It works in the address bar.
Blocking it from running in the address bar would block it too from running from event listeners.
It's a limitation of Opera's stack trace.

I wonder, is your userJS ready/secure enough for prime time? I have an interesting (partially implemented) idea, and i'd like to put it into action .

Originally posted by splondike:

I wonder, is your userJS ready/secure enough for prime time? I have an interesting (partially implemented) idea, and i'd like to put it into action

The interface will be the same.
If it's secure enough... well it needs some testing.

hm, I must did something wrong in here
both userJS files placed in userscript folder
userjs added to one userscript file, and zip file extracted to run on local webserver.

and result is:
---
JavaScript - http://localhost:880/game/crossdomain-xmlhttp.html
Event thread: load
Unhandled exception: "Security violation"
---

Should there be communication or not is the question?
I was looking a way to store data somehow to PC, and crossdomain occured to my mind this morning. And have found this thread.
I have examined code in html page, but I see only hacks you mentioned.
tried to rem them to see if it will work - no it doesn't. So I don't get it yet what was the purpose of this quest

I was thinking to use IFRAME object + localhost cgi but with no success.
So I was thinking to see how cross messaging works, ...

As i remmember should XMLHTTP connect/send data only to host where page originates (location), all other tries should result in security violation?

will closer examine code later, not enough time these days

Originally posted by SolarAngel:

Should there be communication or not is the question?

Hum.. I'll have to do more testing then.
Currently i have a tight schedule.

What is approach Opera`s Widget using to perform cross-domain requests?

A different set of security restrictions; it's the same object.

@xErath: Cheers, i'll put my thingo out sometime soon and see what people think.

Requests are always asynchronous

Was thinking to use your approach, but now I was unable to run in in first place I gave up in first 10 minutes. But maybe I will need it for next task I need to acomplish, ...

Could you give me quick guide how to install/set/make it run on Opera 9.02, wish to give full test run how it works!?

---
This is related to Cross Messaging Approach:
I am curious...

I finally made what was I looking for (small code - I love it)
to save cookies on different zone (eg: domain)
using IFrame injection to document with src to document.location.hostname +'.properties'.
wrote UserJS to implement Cross Messaging, everything works but.

But: can I somehow in function force opera to bubble event 'message' (or what would be right word to fire event in window A, that were raised in window B)?
I have tried with: while(this.msgResponse==null){};
but it blocks execution of script in endless loop, this.msgResponse never get value until execution exits function. Trying to make synchronious talk between pages/domains.

Good thing is that my whole script is asynchronous (eg: user can change values while it is executing), so this helps me - but I would love to have synchronous communication

I am just interested is it possible!?
or I am searching for immposible

[SA]

Originally posted by SolarAngel:

Was thinking to use your approach, but now I was unable to run in in first place I gave up in first 10 minutes. But maybe I will need it for next task I need to acomplish, ...

Place both file in your userjs folder.
In any other User Script, do var foo = new opera.XMLHttpRequest();, and use foo at will.

Originally posted by SolarAngel:

while(this.msgResponse==null){};

This can't work for a very simple reason. Javascript always executes in a single thread, to prevent concurrent access to resources.
So your script blocks competely, in that cicle.
The proper way is to listen for the postMessage event.

Originally posted by xErath:

Place both file in your userjs folder.In any other User Script, do var foo = new opera.XMLHttpRequest();, and use foo at will.

nice I just have code that uses XMLHttpRequest might easily replace, to check how it works
also cross domain - it seams all my problems come from Cross Domain Security - I know, exploits, I hate them too. but where it will end if we start to forbid everything

---

Originally posted by xErath:

Originally posted by SolarAngel:while(this.msgResponse==null){};This can't work for a very simple reason. Javascript always executes in a single thread, to prevent concurrent access to resources.So your script blocks competely, in that cicle.The proper way is to listen for the postMessage event.

I have noticed that today
no timers, no events, no nothing will fire until previous execution is done.

Tricky question: Can I set listener for 'load' on IFRAME that is cross domain, I don't have any detection so far is document loaded or not (eg: page inside IFRAME). I have created timeout event with 5 seconds after IFRAME injection (to make sure it is loaded) but it would be nice to detect load event to be sure / also to speed up execution if loaded

Originally posted by SolarAngel:

Can I set listener for 'load' on IFRAME that is cross domain,

No. Trying to do anything with a i?frame in another domain will fire exceptions.
The only way is to properly detect your pages with a user script and exchange messages.

hm well that is a problem I use IFrame to navigate to 'http://[page source domain].killerscript.storage/' and most of time it works fine, but sometimes just doesn't want to run userjs with no visible reason (eg: no errors, valid @include path, ...).

hm Just got an Idea, I will set in storage UserJS to post message to parent, hm or I might not do that (security ?). oups lol I ment window.parent.parent lol - under page is frame, and UserJS is executing in parent window to that frame LOOOOL don't ask.
have something like this:

[domain1] (framesets/no body)
+-Frame1 (hm not even sure on what domain and what it does LOL, invisible)
+-Frame2 [domain2] framesets/no body - loads my UserJS, all other page on this domain)
  +-Frame2.Frame1 (left menu)
  | +- Frame2.Frame1.IFRAME ([FAKE domain3] Data Storage)
  +-Frame2.Frame2 (site content, pages, ...)

so bottom line is Frame2 controls everything.
rule #1 must not set cookies there, so I am using Frame2.Frame1.IFRAME for tast task. and IFRAME is in Frame2.Frame1 cause this frame is more static, and can't add IFRAME to Frame2 (no body, append child, innerHTML, ... not even sure is there any other way but not to open another window )

now my approach is call post message for read Cookies on IFRAME from fake domain, one of the arguments passed in is string containing callbackproc, and set timeout for recheck msgResponse (bad idea!)
now better idea would be to get some event something anything even from listener for message that IFRAME is running, rather than timeout check.
will try with postMessage to parent.parent if possible - but I have 'no way hose' instinct everytime I think something for Opera.

ok will try this id I don't make it in 20 minutes I will leave and try your Asynchronious Cross Domain XMLHTTP could I make it run, with my code (by just replacing it in Sack class) cu l4t3r

Originally posted by SolarAngel:

ok will try this id I don't make it in 20 minutes I will leave and try your Asynchronious Cross Domain XMLHTTP could I make it run, with my code (by just replacing it in Sack class) cu l4t3r

I didn't tested it in a page with frames... I might not work.
But it's easy to fix that.

s=document.createElement('script');s.setAttribute('declare', 'declare');s.src='http://example.org';this.responseText=s.text;

Nice try Hallvord but the text inside the script element will remain empty.

What does the declare attribute do wrt a script element? What would a user interaction be on an element that is not able to be displayed/rendered?

Using a script element as Hallvord suggested doesnt seem to work unless the script is added to DOM and it wont work more than once for different urls using the same script element

Originally posted by Stoen:

What does the declare attribute do wrt a script element?

I delays the script execution. When to, I don't know. Ask Hallvord.

Originally posted by Stoen:

Using a script element as Hallvord suggested doesnt seem to work unless the script is added to DOM

It works with BeforeScript if added to the DOM.

Originally posted by xErath:

I delays the script execution. When to, I don't know. Ask Hallvord.

Yeah I saw that but was wondering what kind of user event could interact with or otherwise effect a script element...

hello

i downloaded from this forum page :

aaa-greasemonkey-functions.user.js
a-lib-stacktrace.js
a-lib-xmlhttp-cd.js

Test (url) works.

But a script of my friend (developed for firefox) works under windows 2000 - Opera 9.10, but not under
ubuntu 6.06 opera 9.10.

Any ideas ?



form = document.forms [1];

...snip...

try {
if (!GM_xmlhttpRequest) {
alert ("Please update your Greasemonkey installation!");
} else {
form.addEventListener ("submit", lookup_responsible, use_capture);
}
} catch (e) {
var banner = load_field ("banner");
var msg = document.createTextNode (" Greasemonkey support missing - nickname lookup won't work!");
banner.appendChild (msg);
}

Do you get any error in the console or any kind of notification ? Can you attach the scrtipt here ?
Have you followed these steps to install the scripts ? http://userjs.org/help/installation

Originally posted by xErath:

Nice try Hallvord but the text inside the script element will remain empty.

Sorry, forgot adding it to the document somewhere. Should work, I've done this myself. You could always remove it later :-p

Originally posted by Stoen:

What does the declare attribute do wrt a script element?

It's actually a part of Voice / XML Events support, (xErath posted a link to the definition of the declare attribute on OBJECT which is a part of HTML4 I think noone understands :-p ). <SCRIPT declare> basically means the script is not evaluated.
Note: Please don't use declare on SCRIPT elements ever. It was somebody's stupid idea to reinvent function declaration and event assignments as declarative markup. It was a bad idea and it causes compatibility problems for other UAs just like IE's equivalent <SCRIPT for="..." event="..."> does. Just forget declare exists, please #-(

@hallvors: Now I (think I) understand your hack. Nifty. But isn't this more like an example of a workaround rather than a break of xErath's script? If not I'm confused

But as far as I know, s.text does not work (anymore?). I saw the bug on [HTMLScriptElement].text and tried to reproduce it a few months ago... wasn't able to.
Does <script src="some.url" declare/> prevent BeforeEvent or BeforeScript or BeforeExternalScript to fire?

Btw. just tried, <script src="javascript://do stuff" declare/> still executes.

@xErath: I didn't yet test the test case, but if it works in the adress bar,
window.location.replace("javascript://malicious stuff")
might work, too (but this might possibly be catched by the "BeforeJavascriptURL" event).

I updated the script, initially fixed a small bug, thanks to charleso, but then I realized it needed some security overhaul. Lets just say that anyone could steal many of the data kept by user scripts which used this script with sensitive information like usernames and passwords for the http requests.
The link in the 1st post points to the updated script.

If you find any problems please report.

Anyone successfully use request.setRequestHeader? I keep getting the WRONG_THIS_ERR

(function mySpace_gmUpdate() {
  GM_xmlhttpRequest({
    method: 'GET',
    url: 'http://home.myspace.com/index.cfm?fuseaction=user',
    headers: {
      'User-Agent': 'Opera/' + opera.version() + ' (Greasemonkey emulation)',
      Accept: 'text/xml'
    },
    onload: function(responseDetails) {
      // Update Message/Friend Request/etc indicators
      GM_log('response = ' + responseDetails.responseText);
      var html = responseDetails.responseText.replace(/[\n\r]/g,''), i, a;
      while((a=/id="(\w+)" class="show/g.exec(html))!=null) {
        for(i=1;i<a.length;i++){
          document.getElementById(a[i]).style.display = "block";
        }
      }
      RegExp.lastIndex=0;
      while((a=/id="(\w+)" class="hide/g.exec(html))!=null) {
        for(i=1;i<a.length;i++){
          document.getElementById(a[i]).style.display = "none";
        }
      }
    }
  });
  setTimeout(mySpace_gmUpdate, 15000);
})();

GM_xmlhttpRequest definiton:

function GM_xmlhttpRequest(options) {
  var request = new opera.XMLHttpRequest(),
      validEvents = { onload: null, onerror: null, onreadystatechange: null };
  
  function addXMLHttpRequestListener(request, eventName, callback) {
    request[eventName] = function(event) {
      var responseDetails = {
        responseText: request.responseText
        ,readyState: request.readyState
        ,responseHeaders: (request.readyState == 4 ? request.getAllResponseHeaders() : '')
        ,status: request.readyState == 4 ? request.status : 0
        ,statusText: request.readyState == 4 ? request.statusText : ''
      };
      callback.call(null, responseDetails);
    }
  }
  
  for (var eventName in validEvents) {
    if (options[eventName])
      addXMLHttpRequestListener(request, eventName, options[eventName]);
  }
  
  request.open(options.method, options.url, true);
  
  for (var header in options.headers) {
    GM_log('options.headers[' + header + '] = ' + options.headers[header]);
    request.setRequestHeader(header, options.headers[header]);
  }
  
  request.send(options.data);
}

which leads to this error (with debug above it):

JavaScript // debug 
  Unknown thread
  options.headers[User-Agent] = Opera/9.20 (Greasemonkey emulation)
JavaScript - http://home.myspace.com/index.cfm?fuseaction=user&Mytoken=hvarpy,iamaFearkthx
  User Javascript thread
  Error:
  Unhandled exception: [Object InternalException]
  code: 7
  message: WRONG_THIS_ERR
  Backtrace:
    Line 345 of User JS script 
      XMLHttpRequest_setRequestHeader(request, hdr, val);
    Line 136 of User JS script 
      request.setRequestHeader(header, options.headers[header]);
    ...
    removed lots of repeated code from above

Any clues?

I think that xErath forgot to call 'call' method in line 345:
XMLHttpRequest_setRequestHeader(request, hdr, val);

And btw, I've just now grown up to test this script and I've noticed that testcase isn't working out of the box. Your calling xml() function that is nowhere to be seen.

Originally posted by fearphage:

Anyone successfully use request.setRequestHeader? I keep getting the WRONG_THIS_ERR

baw! I had it fixed on my userjs folder, but forgot to upload

Originally posted by d.i.z.:

Your calling xml() function that is nowhere to be seen.

It's i my 3rd post.

Hi @ll,

the cross domain script would be really useful... If I could manage it to do what I want...

I have 2 problems:
1. one seems to be the same one that fearphage is dealing with: Calling setRequestHeader does not work. But at least there is an error message so I might be able to figure out the problem.

2. I boiled my code down to the following sibnce it was not working:

var req =new opera.XMLHttpRequest();
req.open("POST", 'http://www.google.de', true);
req.onreadystatechange = function()
{
alert('pft');
}
req.send('dummy=dummy');
alert('done');

very simple and the 'done'-alertz shows up. The 'pft' won't ever show up :-(.
The strange thing is, that I think I once got this part working and while continuing to develop on another day it suddenly refues working.
There is no error message in Opera's Error Console.
Does anybody have any idea what might eb going wrong??

EDIT: Well.. IK just got an idea: I limited the @include-directive to the url I needed to use it with.
This was only one frame of a page. It does not seem like a good idea, since removing the include-directive solved the second problem...
Buty y did it happen anyway? The opera.xmlhttpRequest was defined after all...

EDIT2: Ooookkk... I think I really did understand the problem now:
The a-lib-xmlhttp-cd.js needsd to be included on bothe the page I am calling from AND the page I am calling with ajax.
Though it is suficient to include the a-lib-stacktrace.js only on the page I am calling from...
I hope this will help somebody out there :-).

EDIT3:
Still a Question:
My Script seems to be executed twice. Since I am setting a boolean to check whether my script has already been executed preventing double execution I conclude that the whole frame is loaded twice.
Is this a result of this script? Can I prevent it?

Originally posted by yankeeCGN:

I have 2 problems:1. one seems to be the same one that fearphage is dealing with: Calling setRequestHeader does not work. But at least there is an error message so I might be able to figure out the problem.

That's fixed. Redownload the script.

I don't think this UserJS is working right in the latest weekly builds (Win 8746, Unix 598, etc).

Can anyone confirm?

Indeed it got broken, because now Opera fires a few more exceptions when accessing some xmlhttp properties before readyState 3, which is expected.
Although I fixed the script for me, there seems to be a new regression in the current weekly, and Opera completly locks up, 100% unusable, and has to be killed when using the script. It seems the xmlhttp request done in the hidden iframe cannot reach readystate 4 for some reason...
It's reported.

I've just uploaded a new version which fixes the problem.
Let's hope it works fine.

I have used these scripts to port the greasemonkey script Google Reader Subscribe Button to Opera.

I'm trying to use your script (seems pretty useful so far :-) ) and after working for a while, I now get some weird error...

Opera just logs in the error console:

JavaScript - <myURL>
Event thread: load
Unhandled exception: "opera.StackTrace module required for opera.XMLHttpRequest module"

Both scripts (stacktrace and xmlhttp) are included...
Is it some bug? (but why did it work before?)

You have both scripts in your global User JavaScript directory?

Yes, I have.

If I insert a alert("foo"); at the beginning of a file some weird alert box I can't explain appears, telling me "foo" ;-)

So I'm pretty sure the Script get's loaded.

What version of Opera are you using?

hum.. both scripts work fine for me in both Merlin (<=9.2) and Kestrel (9.5), no syntax errors, nothing.

Perhaps deleting the scripts, and downloading new copies of them, might help? Just a random thought, at the very least.

Also, is there a specific site that you have been seeing the errors on?

What other UserJS scripts are you using?

The postMessage function was moved up to the window object instead of the document object:


Opera 9.51:
javascript:alert(window.document.postMessage); => "undefined"
javascript:alert(window.postMessage); => "function postMessage() { [native code] }"

http://lists.whatwg.org/pipermail/commit-watchers-whatwg.org/2007/000188.html

Does it work with Opera Kestrel and POST method please?

We can't store data safely, we can't use XMLHttpRequest with Cross-domain easily in UserJS whereas it's possible with Widgets...
Opera's developpers hate UserJS or...?

Does this script still work as intended in Opera 9.5? I have some problems get it running in the new version. Maybe there are some changes required because of the thing Smir has pointed out...

Error:
name: TypeError
message: Statement on line 97: Cannot convert undefined or null to Object
Backtrace:
  Line 97 of User JS script : In function resetFunctionsCall
    		setTimeout.call = parseInt.call = postMessage.call = opera_addEventListener.call =
  Line 325 of User JS script : In function XMLHttpRequestCD
    			resetFunctionsCall();
  Line 264 of User JS script : In function addadvedit_load
                    var xmlHttp = new opera.XMLHttpRequest();

Originally posted by Jan_Rei:

Does this script still work as intended in Opera 9.5?

No, I uploaded a fix.
Refer to the 1st links.

I'm afraid it doesn't appear to be working in 9.52.

It creates the iframe and goes off and gets the data but onreadystatechange and onload do not appear to fire

I even uncommented window.alert("onreadystatechange: "+this.readyState); in the source but get no alerts.

urgh !!
Opera bug, window.parent is unaccessible.