Offline webpage viewing

Offline webpage viewing

It would be really nice to have a feature similar to internet explorer's offline mode where a user can select certain pages in their bookmarks to be caches for offline viewing. You can set it to gather all pages up to 3 links deep which is nice because on a new article like slashdot, you'll get just about everything.

+1
I'd like to go beyond this.
What about some bookmarked pages that I want to view locally, no matter I'm working online or offline? For example, http://my.opera.com/community/smilies/. Such pages aren't intended to be updated ever more, but, if so, I could hit F5 to store an updated version.
w3.org reference documents could be handled that way, too. It would be great to store documentation locally!
It's quite the same if I manually download all the pages (and attachments), save all the files in a local folder, and later browse that folder to read the pages. But Opera could make it much easier, allowing us to bookmark pages to be stored and read offline as default.
It would save paying network traffic, for those who care about this...

you can do this ! go to File -> work offline. If a page is in cache it'll show up...

Originally posted by xErath:

you can do this ! go to File -> work offline. If a page is in cache it'll show up...

Sometimes it would be nice to better control the cache management. As I said in other threads, I am missing some permanent-cache-on-demand (permanent cache = "archive", maybe that's a better name ?).

This is not the same as simply saving a page, often a page dumped in the filesystem is much farther away than a page on the web. The important thing therefore is that Opera should help me organizing..

Browsing/searching these archived pages can be done via history (let's say these pages will never be deleted from history..)

Even more helpful with pdf and other file types, not just websites! Additional search terms for images etc provided by clicked link names..

(sorry for repeating myself from other threads, but I am trying to put the pieces together step by step.)

Originally posted by Schneemann:

I am missing some permanent-cache-on-demand

I agree.
For example: a Java tutorial. Opera could help me this way: bookmarking the "Table of contents" page, downloading all linked pages under that directory structure (1 level, n levels, all levels) and marking those files to stay in cache FOREVER (until the next time I would like to update them). All time long, as I access that URL, Opera retrieves files from cache (not from network, even online) and signs it for me somehow (a color, an icon...).
It's like some kind of "archive" of web content. I will not have to save those files somewhere else in the HDD. I just save and view them in Opera. Wouldn't it be great?
It seems to me that it needs just a little enhancement to Opera's caching engine and a UI for offline bookmarking.
The ability to store (zip?) those files in order to take them anywhere else would be a plus, but I think it would be more complicated. Opera should convert absolute links to relative links, and I don't think this would be simple.

Originally posted by barulheira:

Opera should convert absolute links to relative links

Why not keep the online links, but associate each one with a cached file via some lookup table? I guess this is how the cache works already, it just needs an extension ..

Originally posted by barulheira:

The ability to store (zip?) those files in order to take them anywhere else would be a plus, but I think it would be more complicated.

+1
The question is, how are the cached files stored on the harddrive? If you copy them, will another Opera installation (on another machine) be able to see them as "Opera archive/permacache" with a special index structure, or simply as webpages/multimedia?

Originally posted by barulheira:

it needs [...] a UI for offline bookmarking.

For the UI, I suggest a new button in the adress bar with a drop-down, like the [<] button has.

[<<] [<-] [->] [>>] [refresh] [permacache]
                              0 levels (1 page)
                              1 levels (6 pages)
                              2 levels (133 pages)
                              all levels on server (210 pages)

The button should create a download item in the transfers tab, that needs a tree-like structure for this purpose.

[+] page x (level 1)

and can be opened like that

[-] page x (level 1)
linked page a
linked page b

Originally posted by Schneemann:

Why not keep the online links, but associate each one with a cached file via some lookup table? I guess this is how the cache works already, it just needs an extension ..

I was talking about zipping the files to take them anywhere else to browse locally. Concerning permacache, you are right, there's no need to change links.

Originally posted by Schneemann:

If you copy them, will another Opera installation (on another machine) be able to see them

I wasn't thinking about taking those files to another Opera installation. Some kind of import/export? It would be a plus. At this moment I care about offline browsing at one single Opera installation. If I want to do it, for instance, at work and at home, I could do it twice, for now.

Originally posted by Schneemann:

The button should create a download item in the transfers tab, that needs a tree-like structure for this purpose.

I don't think that that tree-like structure is needed. It would be a plus, too. Downloading each file one after the other (or however Opera handles it) in the transfers tab would be enough for me at this moment.

Originally posted by barulheira:

Downloading each file one after the other (or however Opera handles it) in the transfers tab would be enough for me at this moment.

If it's a big site with many pages (like the java tutorial) this will totally bloat the transfers list.

Maybe it is better to have a new panel for the permacache? because the functionality is slightly different from the transfers. In the transfers, each download is associated with a position in the filesystem, in the permacache only Opera needs to know where and how the files are actually stored.

Still thinking about that ..

Ja, das stimmt!
Yes, we are talking about page download, not about file transfers.
Currently, Opera displays page download status on the progress bar. It shows how many files (attachments - frames, images...) remain to be downloaded, and not much information about them.
Opera could use just the same engine to download files to browse offline. It could handle each linked page as it were a frame, for example. It would be enough for me, and I think it would be easy for Opera developers to implement. Further, a panel showing the tree-like structure would be interesting, too.
Just my humble opinion.

Originally posted by barulheira:

it would be easy for Opera developers to implement

Poor devs we do the pleasant job of UI design, and they have to hack it in!

Oder lese dies auf Deutsch:
K K K K K K!

Hey, now you are talking here about a fully-fledged OFFLINE browser, don't you?

Why not use one instead, OPERA is not designed for that, M/B if they have idle time they will delve into this, in the (farer) future ...

The task is much more demanding than you could think of at a first glance, you may have a look into "Offline Explorer Pro" from METAPRODUCTS e.g. or the like ...

Maybe I should use another offline browser while Opera doesn't provide this feature. But why not put this in this wish-list?
BTW I love browsing with Opera, and browsing offline pages using Opera features would be great!
I don't think it demands too much from developers. The basics are:
- add the feature of marking some cache files to never be deleted (maybe adding some "expires" date far in the future);
- add some button to setup the way download should dive into the levels;
- download all related links as if they where frames of the main page.
Opera devs could say if it's that difficult.

Heya barulheira!! Du dreckelije duitsen
Auf Deiner page steht aber du bist n Brasilianer .. na wahrscheinlich irgendwann mal eingewandert ?

Originally posted by barulheira:

I don't think it demands too much from developers.

You should never say a task is not difficult, unless you were successful doing it yourself! I imagine it is not at all trivial to change the cache management engine.
However, I would be happy with some basic support of the permacache for now.

By the way, do you prefer the name permacache or page archive? Or something different?

Originally posted by HaJotKE:

Hey, now you are talking here about a fully-fledged OFFLINE browser, don't you?
Why not use one instead, OPERA is not designed for that

I can tell you why not:

- I want to do offline browsing with all the features that Opera has.

- For me as a user, offline and online is not much different, html-pages look the same from the cache or the web.

- I use to switch between online and offline all time long, and I don't want to switch between two programs for that.

- I have to use Opera anyway to get the pages from the web, so why should I use another program to read them? And how will Opera tell the offline browser where the important pages are??

- I am using (abusing) Opera as an offline browser already, by just leaving the tabs open and never shutting down. This is the main reason for my awful tab salad, and I suspect I am not alone with that!

Great! Totally agree!
Bald sende ich dir ein PM. Mein Deutsch ist etwas schämlich (sp?)...

Originally posted by Schneemann:

By the way, do you prefer the name permacache or page archive? Or something different?

Permacache sounds techie. Page archive is better. But English is not my native language. Let Opera's staff decide (who are Norwegians... )

Originally posted by barulheira:

Let Opera's staff decide (who are Norwegians... )

At least, most of them ... I believe.

It's a MULTINATIONAL company AFAIK, with employees from many countries!

Yeah... just kidding.

I understood it that way ...

Back to topic:
... but PermaCache is more general ...

It's somewhat contradictory, however. Cache AND permanent?

How about: BackupCache?
or simply BACKCACHE!

That would do it ... IMHO.

I don't like the word "cache", because it's techie. "Cache" is a word used to setup the way temporary internet files (IE's!) should be kept, and is used to give technical support ("Could you please clear your CACHE?"). Non-tech users (I think my mom, my girl, some friends of mine) don't even know what is cache, but they would like to store a bunch of pages for reading later while offline.
I like to say it would be a way to archive (store) web pages. For this reason I prefer something like "page archive" (by Snowman)

I understand, but CACHE is used all-over in OPERA: Help, Preferences etc.

And BTW, it's not only limited to store files temporarily, and why not explain it carefully and understandable?

But I am a "techie" myself

M/B "BackArchive" is somewhat easier, because "PageArchive" is too limited, IMO

But I fear, this is 'eine typisch "Deutsche Diskussion"'

Ja wohl!
I agree, cache is not only for temporary files.
For non-techies: BackCache, PermaCache, makes no sense. For us, techies, it means what it does.
So let the Norwegians decide. (If they really want to accomplish our wishes...)

P.S.: So, was ist "eine typisch "Deutsche Diskussion"'? Ich bin kein Deutscher... warum ist das typisch?

They will decide it anyway, if there will be something to decide some day ...

OFF TOPIC:
It's so typical, because it's just so nit-picking about something not even existing as a real problem ...,
really academic and only debating a pure matter of principle ...

That's what Germans tend to do in many situations; call it some lack of practicality or simply "missing the right joys of life"!

Bob Dylan comes into my mind: "Never think twice ..."!

I thought it should be something like this... Anyway, I wanted to ask first.
Thanks for the explanation.
So there's something "Deutsch" in my soul, too.
CLOSING OFF TOPIC

permacache workaround:

Save the page, open it again - this will add the disk version to your history!
The first two things can be automated with a button I guess (though I don't know how to do..)

Links between pages stored this way will usually not work, however..

+1 for adding offline browsing features to Opera.

Being able to use Opera for offline browsing would be an awesome feature - especially for the dialup brigade (a not insignificant number of web users). I don't always have a net connection, and often have to piss about downloading important information manually when I know I won't be able to go online. There are many programs that don't include documentation, and the online documentation is not available in a single archive to download and peruse locally, so having the ability to grab the lot in one go (and refresh/update very easily) would be great.

If this could be integrated into the existing history viewer that that would be pretty cool.

However, to make this feature really useful, you would need to provide the user with plenty of configuration options (download images, stop at certain page/image size, number of pages, total size of pages download, download files, file types to get, max size of files to get...) so it doesn't seem particularly trivial anymore.

I really like the permacache (for images) idea. The problem is how does Opera know which images are worth keeping and which are not?

For example, if I visit news.bbc.co.uk, there are lots of images, but only a few of them appear every visit to the site, the rest are changed almost daily. How can Opera know which images to store permanently, and which to store temporarily?

Also, which sites does Opera create permacache archives for? Just favourites? Sites that are selected by the user when the site is visited?

What about making the permacache work like Winamp Media Library?

- Whenever you save a file to your disk you may enable a checkbox to make Opera "watch" this folder for the media library..
- In the file browser, each folder's context menu gets "Opera -> watch in media library"
- Of course you can manually edit the list of folders watched.
- The media library is not initialized on Startup, but in the background while you are surfing, so you don't have to wait for it.
- A sidebar panel lets you search in the media library.
- Additional search terms are generated when saving an image file (names of hyperlinks you clicked, image captions on the specific website).
- Instead of opening a www link, you may tell Opera to have a look in the Media Library first. A good indexing scheme might enable Opera to show directly in the link's context menu whether this page exists in your media library.
- Whenever you look at media in your filesystem, the adress field shows a special color

Wow! This is a lot of enhancements! Wouldn't it turn Opera browser a little "heavier"?
Maybe it would be great for many users. I myself like the way Internet Exploder handles offline bookmarks (favorites): not many options, just enough to store content offline, and using just the same browsing/caching engine.

Originally posted by AndyP:

However, to make this feature really useful, you would need to provide the user with plenty of configuration options

I could live without most of those options. For instance: whether or not to store images could lay on whether imagens download is turned on/off/cached only. I would like to tell where I want to retrieve images or pages from: same directory/subdirectories, same server, same domain, or all (just n levels deep). Ad images usually come from different servers or domains. Thus, everything "near" the main page uses to be more relevant. It seems smart enough to me. Options like file types to get, max size of files to get, would be very useful, of course.

Originally posted by Schneemann:

However, I would be happy with some basic support of the permacache for now.

Me too.

hehe again i feel encouraged to recover that old thread..

I just noticed the perfect example where offline browsing features would be VERY useful. It is campuslife.de. And that's the way I want to use it:

I connect via dialup, then have a look at all the "Alle Infos" links (each one opens a popup). Then I disconnect and phone a friend, to tell him about the cool parties. The problem is, I can't use the phone and the internet at the same time, this means I can't open the popup infos again.

The most elegant solution to this would be a cache-on-demand: mark all the links in the links panel, and then choose "cache for one week" from the context menu (which is currently impossible). This would allow me to view these pages though I'm disconnected.

On the other hand, it's a bit mysterious to me why Opera doesn't add these pages to the cache automatically once I open them ..

but surely you can leave the Opera windows open? Are they strange layer popups that aren't treated like regular windows or something?

Still, I agree that it would be a nice feature to have.

Originally posted by AndyP:

but surely you can leave the Opera windows open?

Sorry but how should I find one of 15 popups in the big soup of tabs? It would be more convenient to have them open on clicking the link.

Fine!
Does "Work offline" not work for opening those popups?
Opera should open those popups offline, at least some time after you opened them, as long as they are in cache - unless the server told Opera not to cache them.
Can you check whether those popups are cached? (Look at cache4 folder.)
Test with Opera 8.5x. Opera 9TP1 cache handling isn't trustworthy right now.
If you cannot open those popups offline, and if the server isn't telling Opera not to cache them, it may be a bug.
Well, how could we check what is the server telling Opera to do? http://my.opera.com/community/forums/topic.dml?id=111920 While Opera doesn't provide this information for us, we have to try with some other applications. Maybe Firefuchs...
So, still +1 for your idea.

Now I know why these pages don't go to the cache - they are javascript (that's what the status bar says). FF doesn't cache them too. It's the same with dynamically created pages from bahn.de, like this one ROMA->OSLO.

It would be very helpful if the devs find a good solution for that. I can't tell how this would work technically, because I don't know about the handling of dynamic pages inside the browser. As it is now, the only solution is keeping all pages open (which I am not very happy with).

We have Obook now, that does the job of a "cache-on-demand" very well.

If the Obook pages were stored in a special format, it could be used by Opera as an extended disk cache for a "simulated" offline browsing experience ... would that make sense ?

Maybe this would require an "official" version of Obook.

+1 for offline browsing

Schneemann: could you say something more about Obook?

How about www.webaroo.com?

I never realised that offline browsing was possible in Opera. Why? Because if I go to offline mode, and open a page, Opera alerts me that's I'm offline. When I click 'No', to stay offline, the page I tried to load does not load! But if I type it again, it loads! Bad UI here, guys. If I click No to stay offline, the page should be loaded from the cache.

Also I just tried to submit this post with Offline mode on, and I got a '404' type page - if the page is not in the cache we should be alerted in some way.

Originally posted by cenebris:

Schneemann: could you say something more about Obook

See my signature! Or this link, if my signature should change.

Originally posted by cenebris:

How about www.webaroo.com?

This looks very promising! My idea with Obook is quite similar.. instead of downloading the wikipedia as a webaroo package, you would tell Opera to store the wikipedia in a special Obook folder (this would take some time, of course), with occasional updates running in background with low priority, and use this data as an extended cache to simulate the full browsing experience in offline mode.

Of course you can use webaroo instead (maybe I should try it), but I don't want to miss the cool Opera interface, shortcuts etc when browsing this offline stuff. And, the webaroo packages are quite limited.

Originally posted by svivian:

I never realised that offline browsing was possible in Opera. Why? Because if I go to offline mode, and open a page, Opera alerts me that's I'm offline. When I click 'No', to stay offline, the page I tried to load does not load! But if I type it again, it loads! Bad UI here, guys. If I click No to stay offline, the page should be loaded from the cache.

Also I just tried to submit this post with Offline mode on, and I got a '404' type page - if the page is not in the cache we should be alerted in some way.

The error page is rather useless, and needs to be improved. See http://my.opera.com/community/forums/topic.dml?id=152371. I don't know which kind of alert you imagine.. i hope not a popup dialog. Maybe in offline mode the links should get a special color and tooltip if the target page is not in cache... or be non-clickable.

Would be great if there was some smart offline mode (default) that would automatically show cached files/pages if there were connection problems / user went offline and it would be almost invisible for user (no action required, only small icon or text). Plus it would be really great to choose which, when and how cache certain webpages (+ sublevels)

I think more control over caching would be nice. I don't really like disk cache, but if I could cache just my favorite news sites on the disk for ~instant load it would sweet.

+1

I am happy to see people ask what I asked many times!

http://my.opera.com/community/forums/topic.dml?id=162922

+1

It's sad that nothing in this matter was done. I could really imagine using offline browsing on notebooks...

+1

hhnn, I think you can do this with obook...

+1


for any type of Cache recording

+1

+1 Very good idea!

Seems to be majorly improved in Opera 10: you really can work offline, and pages which are in cache just load.

However it's too seamless when offline mode is not selected, but connection fails. Opera loads from cache silently - no indication that one is offline. My netbook often fails to get itself online after waking up from sleep mode - and with Opera showing cached pages, it can be hard to figure out if one is connected or not. So - add OFFLINE - PAGE FROM CACHE to title bar?

+1