Opera Core Concerns

Originally posted by Philip Jägenstedt:

SVG's own

:heart:Happy New Year 2010
......oooO.............
......(....)..............
......)../....Oooo.....
.....(_/.....(..../......
..............)../........
.............(_/.........
....oooO...............
.....(....)...............
......)../....Oooo.....
.....(_/.....(....)......
...............)../.......
..............(_/........
....oooO...............
.....(....)...............
......)../....Oooo.....
.....(_/.....(....)......
...............)../........
..............(_/.........

Hmm, i have 100% cpu usage on the "9elements HTML5 Canvas and Audio Experiment" and not very smooth, (but it looks pretty simple). But this is because canvas element is very very big. After chaning size of opera window to be smaller, it runs extreamly smooth (but still 100% cpu usage).

BTW. Canvas element on this page isn't resized when i change size of Opera window. It is needed to reload page.

I also have many "mmap() failed: Cannot allocate memory" in the console of "Support Tester" page.

I also think that sound isn't mixed when two or more video is on the one (or multiple) pages.

Anyway, grate work. Many examples works very smooth. Thanks!

When i play the same OGG directly in the mplayer it is slightly smoother, with better sound synchronization. (even with preloaded files into the opera cache, and proxy cache, 10MB/s, or from disk)

Edit: tested it using mplayer side-by-side with opera. Opera have some problems. Image freezes sometimes, then goes quickly to synchronize. Even when it plays continuously it have only about 10 FPS. Mplayer doesn't drop anything (25 fps). CPU usage about 25% (with both opera and mplayer playing!). Audio is playing perfectly.

I also tested it sidebyside with Totem (which uses gstreamer). Totem is much smoother than opera (again 25% CPU with both opera and totem). But totem also have from time to time some problems similar to opera (like freezing and then quickly playing to synchronoize to audio and time). But just after synchronization it runs with full framerate (25 fps), just like mplayer.

good start into the new year...can't wait to see this evolve. for some reason the video and canvas performance thing did nothing on my machine (win/3186) though.

http://files.myopera.com/Tamil/Smilies/ThumbsUp.gif -

Just wanted to add something more. It is about quick start. I doesn't see any buffering. Just loaded page, i hit "Play" button, and few miliseconds later I see moving video and hear audio. Amazing! In most flash player i would wait about 10 seconds. Here in Opera I doesn't wait even 1 second.

I would say http://9elements.com/io/projects/html5/canvas/is one of my favorites as it also follows the mouse

Cool. It took so long that I almost forgot about it!

And also congrats on getting engaged.

If we have a more complete installation of GStreamer already installed on OSX/Windows (say, because we're using GStreamer in Pidgin for voice/video), will the demos that need MPEG-4 work, or does Opera use it's own minimal-GStreamer only?

Originally posted by movax:

I also have many "mmap() failed: Cannot allocate memory" in the console of "Support Tester" page.

Which of the samples were you playing when this happened? Was it MP3? I think this happens sometimes with FFMPEG decoders... will add this to list of bugs.

Originally posted by movax:

I also think that sound isn't mixed when two or more video is on the one (or multiple) pages.

Internally we have one pipeline with one autoaudiosink element per video, which is similar to just starting two instances of totem. It's quite possible that this doesn't always work perfectly depending on the audio output method (OSS/ALSA/PulseAudio/etc) actually used. Are you just hearing one of the videos or is the sound distorted somehow?

Originally posted by movax:

When i play the same OGG directly in the mplayer it is slightly smoother, with better sound synchronization. (even with preloaded files into the opera cache, and proxy cache, 10MB/s, or from disk)

It is a known issue that the video is slightly delayed. Unless Opera is using 100% CPU there's no reason it shouldn't play at 25 fps though, so that's clearly a bug. This is likely to behave slightly different on different platforms too, but again I'll add this as a new bug.

Thanks movax!

So, correct me if I'm wrong on this one, but as I understand it, right now anything other than vorbis/theora will not play on windows?

Why just don't use windows directShow filters that are installed, like any other media player would do?

Originally posted by BigBrownChunx:

If we have a more complete installation of GStreamer already installed on OSX/Windows (say, because we're using GStreamer in Pidgin for voice/video), will the demos that need MPEG-4 work, or does Opera use it's own minimal-GStreamer only?

Since there is no standard distribution of GStreamer for Windows, we can only use our own. But regardless of platform, for any plugins to work they must be linked with the actual GStreamer library being used, we can't "mix and match" since our minimal GStreamer is missing quite a lot of things.

In short, expect Mac to work the same as Windows.

In using GStreamer, it is not our intention to support every format under the sun and doing that wouldn't be very useful since very few of them work cross-platform and cross-browser. The ability to use arbitrary codecs on Unix is a kind of "accident", a little something to play with if you will.

Originally posted by malsumis:

Why just don't use windows directShow filters that are installed, like any other media player would do?

The main problem with this approach is that many DirectShow filters are written in such a way that they can only read from a local file, since the DirectShow framework makes this a lot easier than the "right way" of streaming input from a upstream filter. The net result is that we can't play a partially downloaded resource, which is rather useless.

What is left is MPEG-1, some old AVI formats, etc, which doesn't seem worthwhile supporting, especially as it would be Windows-only.

Originally posted by foolip:

The main problem with this approach is that many DirectShow filters are written in such a way that they can only read from a local file, since the DirectShow framework makes this a lot easier than the "right way" of streaming input from a upstream filter. The net result is that we can't play a partially downloaded resource, which is rather useless.

What is left is MPEG-1, some old AVI formats, etc, which doesn't seem worthwhile supporting, especially as it would be Windows-only.

Sucks. So, are there no plans to implement mpeg-4? Apparently Firefox only supports ogg formats, but safari and chrome can deal with mpeg-4 also, and safari with it's proprietary qt format.

Google has youtube, so they won't make it any easier to adopt ogg as the standard. Maybe it would be wise to support mpeg-4 in some form?

@malsumis: MPEG-4 is a good standard, but it's not open in the same sense as HTML, you have to pay to use it, not only if you include it in your software, but even to distribute content.

I can only speculate, but I don't think Google would offer more than $100 million for On2, the company that enabled the Theora project, if MPEG-4 was the solution to all their video needs.

Originally posted by borg:

@malsumis: MPEG-4 is a good standard, but it's not open in the same sense as HTML, you have to pay to use it, not only if you include it in your software, but even to distribute content.

I can only speculate, but I don't think Google would offer more than $100 million for On2, the company that enabled the Theora project, if MPEG-4 was the solution to all their video needs.

I knew that mpeg-4 needs to be licensed. Didn't know that google bought on2. Interesting. The important thing is what will google
do with on2? Will it go the Microsoft route? Youtube supports mpeg-4 only right now, if they were serious, wouldn't they make it ogg-friendly? Dailymotion has some stupid mozilla agreement and only let's you demo their html5 service with firefox beta something.

And the rest of the web is basically stuck on flash, because right now basically no one cares about html5, even though you can gracefully degrade to flash if you're just a little bit on the smart side.

Originally posted by malsumis:

Youtube supports mpeg-4 only right now, if they were serious, wouldn't they make it ogg-friendly?

The impression I got was that they opted for mpeg-4 reluctantly and wanted to go with ogg, but "needed" something "better", so are currently seeking alternatives (hence On2). That's largely speculation based on bits and pieces I've read here and there.

Originally posted by malsumis:

Dailymotion has some stupid mozilla agreement

I think it's less of an agreement and more that the project has stagnated. They released it, announced it saying "we will add support for other browsers soon" (paraphrased) and then left it to die. Or so it seems anyway.

Extra points to anyone who can identify the movie in the screenshot

Is there no chance of adding the components of gstreamer that allow mpeg-4 playback to the windows install? Or would that mean having to pay license fees? And if so how do open source players like VLC support mpeg-4?

Supporting both would allow compatibility with a greater majority of video now using the

Originally posted by Philip Jägenstedt:

Extra points to anyone who can identify the movie in the screenshot

Malan from Disney?

Originally posted by Philip Jägenstedt:

no reason it shouldn't play at 25 fps though

When Vega adds HW acceleration will the FPS increase?

Originally posted by Serav:

Is there no chance of adding the components of gstreamer that allow mpeg-4 playback to the windows install? Or would that mean having to pay license fees? And if so how do open source players like VLC support mpeg-4?

I can't comment on the legal situation or other projects, but Opera can't distribute MPEG-4 decoders at this time.

Originally posted by DanielHendrycks:

Malan from Disney?

No, but there's a Chinese connection.

Originally posted by DanielHendrycks:

When Vega adds HW acceleration will that FPS increase?

It doesn't look like it's a performance problem right now, it's rather a timing problem. Hardware acceleration may help reduce CPU usage though.

I hope there'll also be a per-site option to disable the video or block certain ones.

Originally posted by foolip:

I can't comment on the legal situation or other projects, but Opera can't distribute MPEG-4 decoders at this time.

Fair enough. I was just curious so thanks for the answer.

Timing issues remind me of the problem with MPlayer and MKV in Damn Small Linux; where technically MKV subs were supported, but nobody had done the timing work for DSL's custom core.

Tacking controls on to the face of VLC's existing browser plugin wrapper, would in fact be rather nice. I already open videos in Opera windows, VLC is cross-platform last time I checked. I suppose there are always legal hurdles, though.

svg and video tags are really good ideas keep it up.

Originally posted by lucideer:

I think it's less of an agreement and more that the project has stagnated. They released it, announced it saying "we will add support for other browsers soon" (paraphrased) and then left it to die. Or so it seems anyway.

Someone should learn them how to properly use video tag. Why the hell they are doing any hardcoded browser detection!? Just embeded video element directly, browsers will decide for they own if they are supporting it or not, and if not provide fallback content inside of the video tag body. Eh.

http://camendesign.com/code/video_for_everybody Example how it should be done.

@Philip Jägenstedt

Are you using ffmpeg/ffdshow for the theora video decoding?
http://sourceforge.net/projects/ffdshow-tryout/

If so then you should have no prob any video/audio pretty much.
I assume it depends on what this GStreamer thing allows in ogm files.

Halli supposedly shares his splitter code with those who ask:
http://haali.su/mkv/
That does ogm, avi, mkv and mp4.

Then with flash video's, you could rip the code from media player classic (probably from mpc-hc) for the stand alone flv splitter.
I've only seen that outputted to ax format, I dn if linux can deal with that or not.

If you could get flv files working.
A script that gets the direct link off youtube and puts it in a video tag.
You could disable the flash pugin and still play it back .
That would rock.

Anyways whatever you decide to do.
Good luck to ya.
It's better then flash anyways.

Originally posted by NEOAethyr:

Are you using ffmpeg/ffdshow for the theora video decoding?

No, we use GStreamer, which is a media framework just like DirectShow. We don't mix the two currently, because there doesn't seem to be any benefit in doing so.

I think it was actually possible to use DirectShow codecs in GStreamer, but meh... I haven't really played with GStreamer on Windows in quite some time. Nevertheless, it sounds kind of bad that if a system version of GStreamer is installed, and in the PATH, that it wouldn't be used on Windows or Mac, though it is on Linux...

It is rather trivial to add in detection logic to the installer to find GStreamer and configure Opera to use it, at least, if it is was a script based installer. And of course, if no GStreamer installation is detected, then Opera's own minimal one is installed and used instead.

How about OSS4 support? I find OSS much more useful than Gstreamer.

Originally posted by panzeroceania:

How about OSS4 support? I find OSS much more useful than Gstreamer.

OSS is a sound system while GStreamer is a media framework (like DirectSound and DirectShow). GStreamer may already be using OSS for audio output, depending on your system setup.

What?? No gitweb to access and nicely browse the source tree with?

Do you plan for FLAC or are you waiting for a losseless ogg video codec to go hand in hand with FLAC?

Sorry about that Pharaoh_Atem, we might add that. Our changes aren't very interesting, but for now you have to clone the repository to have a look.

Originally posted by DanielHendrycks:

Do you plan for FLAC or are you waiting for a losseless ogg video codec to go hand in hand with FLAC?

FLAC (and lots of other codecs) are properly detected and used on Unix (flacdec is in gst-plugins-good) if you want to play with it. As FLAC isn't terribly useful on the web due to its size, we probably won't support it on other platforms (keep in mind that every added codec adds to the download size of Opera).

The lossless Ogg video you talk about actually already exists -- Dirac has a lossless mode. However, lossless video seems even less useful than lossless audio on the wider web

Ok, maybe in years we will see it when we can transfer more information faster.

Are you planning on not being reliant on GStreamer plug-ins for Linux? I do hope so because one purpose of codecs is so we do not have to rely on plug-ins.

And interesting thing maybe comming out of CES 2010

New Video File Format Promises to Work on Any Device [CES 2010]
http://mashable.com/2010/01/04/dece-ces-2010/

Originally posted by DanielHendrycks:

Are you planning on not being reliant on GStreamer plug-ins for Linux? I do hope so because one purpose of codecs is so we do not have to rely on plug-ins.

Actually, from my perspective the real purpose and advantage of baseline codecs is that a web master can take for granted that they will be present and supported by the browser. Even with the slightly fuzzy situation right now of two competing codecs (Ogg Theora and H.264), it is still better than nothing in that it is either one format or the other (personally I think it is still quite possible that a final solution might be reached before the new video tag really takes off).

Furthermore, it is not really a problem to be reliant on certain core patent free GStreamer plug-ins (those that provide Vorbis audio codec and the Theora video codec) as these are ubiquitous on Linux, across distros and processor architectures, given their Open Source status and lack of licensing restrictions. Hence they can be made a dependency of the Opera .rpm and .deb packages if needs be, ensuring they are installed by the native distro package manager at the time of install of Opera. This was not a reasonable option for regular plug-ins like Flash for numerous reasons, the two most obvious being licensing issues and support for a limited range of processor architectures.

From the perspective of most end users users, having the package manager deal with installation of the required media plug-ins is the same as bundling the plug-ins. Better in fact in that duplication of libraries is avoided and an advanced user with no interest in video could override the dependency should they prefer not have the files on their system (.e.g. to keep the size down on a micro distro).

As foolip mentioned if other plug-ins are available and the page suggests it, then great, we'll use them .. but .. the browser only expects the standard baseline plug-ins to be available by default. Everything else is just a nice bonus!

Originally posted by foolip:

FLAC (and lots of other codecs) are properly detected and used on Unix (flacdec is in gst-plugins-good) if you want to play with it. As FLAC isn't terribly useful on the web due to its size, we probably won't support it on other platforms (keep in mind that every added codec adds to the download size of Opera).

could it be an optional addition? just as widgets and Opera Unite apps are optional, surely you could have other optional codecs that aren't part of the standard opera install.

Originally posted by panzeroceania:

surely you could have other optional codecs that aren't part of the standard opera install.

This already works (on Linux/UNIX at least) but the only codecs that the web master can take for granted as being present are the the Vorbis audio codec and the Theora video codec.

Originally posted by foolip:

Originally posted by movax:

I also think that sound isn't mixed when two or more video is on the one (or multiple) pages.

Internally we have one pipeline with one autoaudiosink element per video, which is similar to just starting two instances of totem. It's quite possible that this doesn't always work perfectly depending on the audio output method (OSS/ALSA/PulseAudio/etc) actually used. Are you just hearing one of the videos or is the sound distorted somehow?

O hear only one movie, second is running but is silent.
It can be problem with output. My system runns ALSA but also have OSS emulation, so it is possible that it selected OSS. But when i was testing Opera and Totem (or mplayer) in parallel i heared both simultaniously. So it doesn't look it a OSS problem.

I will check which audio output method gstreamer in opera used.
It is definietly not OSS (because i inspected `lsof` and no /dev/dsp is opened), and it is probably PA (because it is so in `gstreamer-properties`), and PA is using ALSA. SO mixing should work.
I'm running Debian.

PS. I just tested it again, in Two detached tabs, and video and audio is working properly (mixing works). Strange.

Originally posted by foolip:

Originally posted by movax:

I also have many "mmap() failed: Cannot allocate memory" in the console of "Support Tester" page.

Which of the samples were you playing when this happened? Was it MP3? I think this happens sometimes with FFMPEG decoders... will add this to list of bugs.

It happens for example on "Video and Canvas performance demo" link.
(it doesn't happen on Wikipedia, it doesn't happen on TinyVid or Video for Everybody, or MetaVid).
It is exactly 100 messages "mmap() failed: Cannot allocate memory" (of course there is plenty of RAM free, but as I have 32bit system, it is probably trying something which will not fit in address space). It happens only on starting of movie (and it plays fine). I'm running Debian unstable :/ But no other player (probably) shows me something similar. It isn't also very easly reproductible. SOmetimes i have this error, sometimes not.

PS. Tested on another computer (with Debian testing i386). No such issue (no mmap messages), or i just can't trigger it.

I also found small bug in zooming webpage with canvas+video:

GO to http://people.freedesktop.org/~company/stuff/video-demo.html and set zoom to 30%. Webpage will not zoom out completly, but it will be 30% height and 100% width, with not proportionally scalled movies. bottom 70% of webpage will be just blank.

100% http://img30.imageshack.us/img30/228/zrzutekranu1u.png
20% http://img177.imageshack.us/img177/2149/zrzutekranu2k.png

Others webpages with video doesn't have problem with zoom/scalling.

Well looking again, the VLC core is pretty darn awesome. More or less compact, and self-contained. Very fast. Many features could be cut, saving installer size. I suppose even a minimal GStreamer implementation will bloat things up a bit though.