Ruari's thoughts

When I am choosing between either tool for script I often find that Wget's biggest advantage is its attempt to do some interpretation of the certain file-types. Not much interpretation I'll grant you, but it is aware of links and references to external files in HTML and in the latest version, CSS as well. This allows it to download a page with all its requisites and also permits recursive downloading. Understanding what links are also gives it the ability to rewrite absolute links to relative links, or change links in other ways, e.g. when pages link to file names that are missing extentions (or that would be illegal on certain operating systems) it can change the file name and match the links accordingly. You can also get Wget to download only new or newly linked files. So if a page changes or updates and you grab it again (with appropriate options), it will only get the new or updated links but not download files that it already has (it compares files timestamps to achieve this). All of this in turn, allows for great site mirroring ability but these features also have other uses as well.

And it is not just page interpretation and recursion, Wget has a number of other nice options that I value. For example, one of my favourites is the 'spider' option, which checks a page exists but doesn't download it. Like other commands this can be combined with recursion to quickly create a map of the directory structure of a site or check what links have changed without having to download lots of content.

Besides Wget specific features, certain operations are done much more logically than in cURL (though this is often also true in reverse!). Consider sending Wget or cURL a URL via a pipe. With Wget you can do the following:

$ echo http://www.example.com/ | wget -i - 

With cURL this is a fair bit harder. A quick scan of the man page might lead you to believe that you could use '--url -' for this purpose but if you try this it will fail because '--url' will not accept '-' representing standard input for this option. Which is odd, because you can use it to represent standard input with options such as the config file option (-K or --config). What's particularly fun about this is that config files can contain other options, including '--url'. Hence the following would work:

$ echo http://www.example.com/ | sed -e "s/http/--url http/" | curl -K - 

Here I have used the sed command to rewrite the URL to '--url http://www.example.com/' and then told cURL to read this as a config file and ... Voilà, it grabs that URL! Granted an easier way in most cases would be to use the xargs command as follows:

$ echo http://www.example.com/ | xargs curl 

My point however being that for this type of operation, you really have to jump through hoops for no good reason.

So what am I trying to say, that Wget is better than cURL? No, certainly not! As I said before cURL has loads of great options entirely missing from Wget: its large list of supported protocols; its ability to upload as well as download; the ability to grab sequences of files (without using recursion), e.g. ftp://ftp.example.com/file[1-100].txt. This list goes on because cURL is an amazing utility as well!

Actually what I am trying to say is this: Both tools are very powerful and whilst they do have a large amount of crossover functionality, differences in project design and goals mean that neither can currently fully replace the other, and anyone who tells you otherwise is either lying or doesn't understand one (or both) or these great utilities!

Ok, now I have got that off my chest I'll go and see if my download has finished. It shouldn't be long now as multi-connection downloads really maximise the use of my connection. Good thing I am also a fan of aria2 as well! :-P