Uploading files to My Opera with cURL
Monday, 12. October 2009, 09:23:18
Since I have been playing around with cURL of late, I started thinking about how to use it to upload files to My Opera's file space. It can actually be done fairly easily, here some examples.
Note: The following examples will work on Linux, UNIX and Mac OS X. Whilst it is possible to achieve similar results on Windows, I have not covered this.
Note: The following examples will work on Linux, UNIX and Mac OS X. Whilst it is possible to achieve similar results on Windows, I have not covered this.
Before you can upload files you need an auto-login cookie. You could extract the cookie information from your Opera cookie store, but it is just as easy to get cURL to generate a new login cookie. Just change 'username' and 'password' to your own My Opera username and password in the following command and then run it:
$ curl -L -c ~/.myoperacookie -d "location=&user=username&passwd=password&remember=1" http://my.opera.com/community/login/index.pl > /dev/null
Next save the following as shell script. Before saving this you'll need to edit the 'username' parts to make it your own My Opera username:
#!/bin/bash curl -b ~/.myoperacookie http://my.opera.com/username/files/addpic.pl -F file=@"$1" -F dir="$2" -o /dev/null file=`echo $1 | sed -e "s/.*\///" | sed -e "s/ /%20/g"` curl -s -I "http://files.myopera.com/username/$2/$file" -e http://my.opera.com/ | grep HTTP | sed -e "s/HTTP\/1.1 200 OK/Upload Successful/" | sed -e "s/HTTP.*/Upload Failed/"
Note: Make sure you make the script executable and save it somewhere in your execute path.
To use the script simply type the script's name (I called my script 'moput', meaning 'My Opera Put'), followed by the name and path to file you want to upload, and the name of the My Opera directory you want to upload to. For example, to upload a local file called 'funny.jpg' from your local '~/Pictures' directory to your My Opera Pictures directory, you could issue the following:
$ moput ~/Pictures/funny.jpg Pictures
Note: If the My Opera directory doesn't exist it will be created. If you want it to be a subdirectory this is possible to, e.g. 'files/Pictures'. If you only mention the local file and don't mention a My Opera directory, it will placed in 'files' but you will be told that the 'Upload Failed'.
Now the obvious questions are, "How is this more efficient than the regular uploader? What if I have lots of files to upload, do I have to upload them one at a time?"
Firstly, for me at least it is more efficient as I prefer work on the command line. It also allows me to upload files from remote machines that I connect to via ssh (hence no GUI). However, I appreciate that many people won't see these as benefits. In answer to the main question, yes you can queue up lots of files to upload, and no it doesn't even require having to make a more complex script. Instead you can just make use of the other tools on your system. Suppose for example I wanted to upload all the .jpg files in the current working directory to My Opera and save them in the remote Pictures/jpegs folder. For this you could use the UNIX 'find' command as follows:
$ find *.jpg -exec moput {} Pictures/jpegs \;
Here is another variant, which would copy all PDF files in the current working directory and one sub directory deep, and save them in in the remote 'Documents' folder:
$ find . -maxdepth 2 -type f -name "*.pdf" -exec moput {} Documents \;
Note: If you are not familiar with the 'find' command, read the 'man' page or consult one of the many online tutorials before generating more variant commands. 'find' is incredibly powerful, meaning plenty of things are possible, including stuff that you might not intend!
As many Mac users tend to shy away from the command line, here is a nice Graphical option, using an Automator workflow.
Before we can create the Automator workflow, we need a My Opera login cookie. This is exactly the same as my early instructions but I will repeat it here for convenience.
To make a login cookie, start the Terminal application and type the following, remembering to change the 'username' and 'pasword' to reflect your own My Opera login credentials:
curl -L -c ~/.myoperacookie -d "location=&user=username&passwd=password&remember=1" http://my.opera.com/community/login/index.pl > /dev/null
Now we need to create an Automator workflow as follows:
- Start Automator
- In "Select a starting point to open a new workflow", highlight "Files & Folders"
- Select "Get content from": "my Mac" & "Use files & folder selected in the Finder when workflow runs"
- Under "Actions > Utlilities" drag "Run Shell Script" to the workflow on the right, below the initial entry.
- Change the "Pass input" option to "as arguments"
- Replace the example code provided by Automator with the following, remembering to change the two 'username' references to reflect your own My Opera username:
for item in "${@}" do curl -b ~/.myoperacookie http://my.opera.com/username/files/addpic.pl -F file=@"${item}" -F dir=files -o /dev/null file=`echo ${item} | sed -e "s/.*\///" | sed -e "s/ /%20/g"` curl -s -I "http://files.myopera.com/username/files/$file" -e http://my.opera.com/ | grep HTTP | sed -e "s/HTTP\/1.1 200 OK/0/" | sed -e "s/HTTP.*/1/" done | sort -r | head -1 | sed -e "s/0/File(s) uploaded successfully./" | sed -e "s/1/Check My Opera files: Items may be missing./"
- Under "Actions > Utlilities" drag "Run AppleScript" to the workflow on the right, below the previous entry.
- Replace the example code provided by Automator with the following:
on run {input} tell application "Finder" display dialog input buttons {"OK"} end tell return input end run
- Select "File > Save as Plug-in" and name it appropriately (e.g. 'Send Files to My Opera'), as a Plug-in for "Finder".
- Quit Automator
To use: Simply highlight the files you wish to upload in a Finder window, right click with your mouse (or Ctrl+click) and choose "More > Automator > Send Files to My Opera". Automator will then start in the background (you can see the status at the top of the screen). When it is done you will get a message telling you if the files uploaded or not. Done!
Of course if all this seems like too much work, you could just use Unite to share files instead!















Espen André Øverdahl # 13. October 2009, 06:48
Ruari Ødegaard # 13. October 2009, 06:59
it-s # 13. October 2009, 19:04
Espen André Øverdahl # 13. October 2009, 19:06
Ruari Ødegaard # 13. October 2009, 19:33
You can actually do stuff link this on pretty much any site that has a file upload option. It sometimes takes a little work to decipher how to construct the POST request, and you often have to include a step or two for authentication, but this is usually more than paid for in time saved later if you use a site a lot. If you so desired you could also write simple scripts to list files or delete them as well.
The document "Using cURL to automate HTTP jobs" over at the main cURL website does a good job of explaining how this all works. Also reading the man page will probably also give you further ideas about stuff you could automate.
Oh, and whilst in this instance I used cURL, I suggest having a quick skim of Wget's manual as well. As I mentioned in my previous blog post, sometimes it is Wget that works best for this stuff.
Dustin Wilson # 13. October 2009, 19:48
Ruari Ødegaard # 13. October 2009, 20:26
To:
This should pick up on any Proxy timeout problems and authentication errors (for example if the cookie expires).
Other keywords could obviously be added as '-e keyword' in the grep command for other failure pages.
There shouldn't be too many false positives as it is not actually the my.opera.com/username/files/ page that is being echoed back to curl on successful transfer but rather a simple '302' page (the option '-L' would have caused curl to follow on to the files page).
Charles Schloss # 14. October 2009, 03:58
I wonder what Tamil thinks of this?
Ruari Ødegaard # 14. October 2009, 05:58
Originally posted by Chas4:
So, you think Tamil doesn't know about automating content posting on web sites? Haven't you always wondered how those first posts are so consistently achieved on the Desktop Team Blog?
Actually in all seriousness, I don't personally know (or care) if Tamil does this.
Either way Tamil is great asset to our community!
Aleksander Aas # 14. October 2009, 06:29
Ruari Ødegaard # 14. October 2009, 07:50
Otherwise, you can still get a Windows version of cURL from links on the bottom of the cURL downoad page and create a batch file.
Having not actually used Windows for quite some time (my home and work machines are all Linux, UNIX or Mac) I can give some hints from my vague memories of Windows, but I can't guarantee any of it will work. I'm almost certain to forget something critical.
That said .. what the hell! Here goes nothing!! Once cURL is installed somewhere in your path you should be able to run the following commands to create your login cookie:
Hopefully this will create a hidden login cookie file in your home directory called 'myoperacookie.txt'. Open it in Notepad to confirm it looks correct.
Assuming that worked, save the following in a text file called 'moput.bat':
After placing 'moput.bat' somewhere in your execute path, you could (in theory at least) run commands like:
Obviously changing "%userprofile%\My Documents\funny.jpg" to whatever file you wanted to upload.
Or if you were already in the correct directory, perhaps:
With regards to replacement for UNIX 'find', or some other way to queue up lots of files, that is beyond my memory without a Windows machine to test with. Indeed I am not even 100% certain that any of the examples I gave above will work at all.
Perhaps some Windows command line Guru could come along and produce real examples. I can't imagine it would be too hard on Windows. Indeed I assume that 'PowerShell' would be a better option than 'cmd.exe', but as I have never used PowerShell, I will not even attempt that!
Tracio # 14. October 2009, 10:55
I've included your command in a function and place it in my .zshrc . I've changed it a little bit so it can batch upload files from a single command. Furthermore, by using zsh's extended file globbing, the use of "find" can be avoided in most cases.
function uploader () {for item in "${@}"; do
curl -b ~/path_to_my_opera_cookie http://my.opera.com/my_username/files/addpic.pl -F file=@"${item}" -F dir="My_directory" | grep -i -e error -e proxy | sed -e "s/.*/\*\*\*Upload Problems\*\*\*/" | sort -u
done
}
Thus a
will batch upload all .jpg files present in your current directory and all subdirectories.
Ruari Ødegaard # 14. October 2009, 11:24
Ruari Ødegaard # 15. October 2009, 07:57
I also need to tell cURL to send (my.opera.com) referer page information to the server so that the file header is returned and not the header for the download page.
I have updated the main blog post with this variant.
Espen André Øverdahl # 15. October 2009, 08:02
Ruari Ødegaard # 15. October 2009, 08:24
You wouldn't be able to reuse anything I have written here but that doesn't matter too much as the Unite Framework would allow you to do the same thing.
However a better option might be a Unite Application called something like "My Opera Files Sync", which would work as follows: You start the service and nominate a directory on your hard disk, then any files placed in that folder are automatically uploaded to your My Opera Files store.
Indeed it should be possible to create a similar Unite Application for My Opera Photos.
Espen André Øverdahl # 15. October 2009, 08:41
Originally posted by ruario:
This, and:Originally posted by ruario:
this would make me a happy man.Great suggestions!
Ruari Ødegaard # 15. October 2009, 10:04
1. My JavaScript skills are not up to it.
2. The current solution (with cURL) does the job for me.
Hopefully someone else will be inspired though.
Ruari Ødegaard # 15. October 2009, 10:06
I might have a look at that if I find a moment.
Tracio # 15. October 2009, 10:29
Item, "MyOpera-Uploader"="Execute program, "gnome-terminal --execute for item in $(find ~/path_to_dir_being_watched -Btime -1d); do curl -b ~/path_to_my_opera_cookie http://my.opera.com/my_username/files/addpic.pl -F file=@"${item}" -F dir="MyOperaDir" -o /dev/null ;done""That will automatically upload all the files present in the directory which have been modified in the last day, assuming that Opera can pass a for loop to gnome-terminal... I can't test the menu item code stuff right now but the for loop works fine.
Needless to say that a cron job (or a launchd plist on OSX) would work too, that way you don't even need to execute the script, it would do everything automatically.
Ruari Ødegaard # 15. October 2009, 13:10
Ruari Ødegaard # 15. October 2009, 18:36
I now have added Automator instructions to the end of the original blog. Once setup, a user can simply highlight a selection of files in the Finder and right click (or control click) to send them to their My Opera file storage area.
Daniel Aleksandersen # 16. October 2009, 06:14
Ruari Ødegaard # 16. October 2009, 06:32
Originally posted by danaleks:
Nice idea but in the example I provided, the username is hardcoded, hence anyone wanting this has to re-create the workflow themselves using my instructions.
I'm sure you could get around this by asking for the username and password only the first time (and them storing these), but that requires something more complex and I feel that now I have done these examples to death.
My intention was more to show what could be done 'fairly' easily not to write an entire application, widget or unite service.
So if someone wants something more complex or with more options, I think they will need to write it themselves.
endless love # 17. October 2009, 06:11
Ruari Ødegaard # 20. October 2009, 20:14
Here is a very simple cURL script example (with no error checking or confirmation of upload):
You would need to configure this as follows:
XXX should be replaced with the value from a 'unite-session-id' cookie, taken from a browser that is logged into the 'File Inbox' Application as a visitor.
devicename should be replaced with the device name for the unite service you are connecting to.
username should be replaced with the My Opera username of the owner of the unite service you are connecting to.
In Opera you could get the unite-session-id cookie value from the Opera cookie manager (in the Advanced Preferences). Though, as the cookie will expire reasonably frequently, you will either need to update this value manually from time to time or write another cURL script to auto-update it for you. Personally I linked my copy of the script to my w3m cookie file. Then I can just re-login with w3m to get the script working again, whenever the cookie expires. To do this, I just need to change the line that sets the 'unitesessionid' variable to:
In case you are not familiar with it, w3m is a handy terminal web browser that I use from time to time on machines with no GUI running. I figure I can recommend it in good confidence because its a great little browser, and it is not exactly direct competition!