getUserMedia and Device Orientation Adventures

Opera a while back had released an experimental android build, with support for getUserMedia and Device Orientation support. The stuff is really really cool, and is one of the major things which will help close the gap between the capabilities of native and web applications on mobile. See the post on the core concerns blog about these new addtions for more information and demos.

Whats so cool about it?

You know how people who say that native apps are always better than web apps on mobile, because native apps can have access to certain device features like the camera, or the gyroscope? Well, thats no longer the domain of native apps anymore.

Demo Time, Part 1: A simple camera app

I made a simple camera app, which uses getUserMedia to access the camera, and the video tag to stream on the screen. There is a button, and upon clicking it, it takes a snapshot of it, and posts it on a canvas element. The first thing I do is:

if (navigator.getUserMedia){
 	navigator.getUserMedia('video', v_success, v_error);
 } else{
 	not_supported();
 }

If the browser supports access to the camera, then the success function is called, in this case v_success(). Its in this function that we take the video stream from the camera and associate it with the video element.

var video_element = document.querySelector('video');
...
function v_success(stream){
 	video_element.src = stream;
}

Its that simple! Now you have a the output of your camera inside the page. But we want to do more. We want to create a button, which upon click, takes a snapshot of the camera stream, and outputs it to a canvas element, thereby taking our picture.

Let’s get the button and add a click event handler:

var button = document.querySelector('#button');
button.addEventListener('click',snapshot, false);

So when the button is clicked, the snapshot() function is called. Here we take the video, and use it to draw the current snapshot of the video onto the canvas using the drawImage() method.

function snapshot(){
 	var c = document.querySelector('canvas');
 	var ctx = c.getContext('2d');
 	var cw = c.clientWidth;
 	var ch = c.clientHeight;
 	ctx.drawImage(video_element, 0, 0, cw, ch);
 } 

This takes the current snapshot of the video element and puts it in the canvas element, thereby taking our picture. Check out the actual demo page, and source code. Keep in mind, this is just a rough demo, and could be polished further, especially in terms of making it look better ... but that wasnt the focus here with this demo this time for me. Also, make sure to view this in a browser which supports getUserMedia, like our experimental build of Opera Mobile for Android.

Demo Time, Part 2: Psychedelic web forms which clear on shake

This is a demo, in which some values in a form are pre-entered, and if you shake the device you have, then it will clear the text entered in the form. It also changes the background color based on the orientation of your device....so you can just move your device around with your hand, and it will change the background color. Pretty cool, huh? It uses just some simple web forms (in my case, some html5 web forms but it could be any kind of web form) and the device orientation events as mentioned in the W3C Device Orientation Events spec.

There are two key events to note here: The deviceorientation event and the devicemotion event. The former, as the name suggests, tells you about the orientation of the device in terms of alpha, beta and gamma values (a bit like the x, y and z x co-ordinates). The latter, once again as the name suggests, provides motion related information on the device. Whenever the device is moved, these events are fired. Now let’s see some interesting stuff to do with them, starting with deviceorientation.

Here, we'll get the alpha, beta and gamma values of the current orientation of the device, and put those values as an RGB color as the background color. Which means, everytime the device is moved, there will be a different color applied to the background, based on the orientation of the device. Let’s see how. First let’s add an event listener to the window.

window.addEventListener('deviceorientation', update, true);

So whenever the device is moved, the deviceorientation event is fired, which calls the function update(). Our update() function looks like so:

function update(event){ document.body.style.backgroundColor = "rgb( "+Math.abs(event.alpha)+", "+Math.abs(event.beta)+", "+Math.abs(event.gamma)+" )"; }

Next comes the part where we need to clear the form if the device is shaken. The devicemotion event has a property called 'acceleration' (for x, y and z axises). We'll just see if the acceleration is above a certain threshold, and if so, we figure that the device has been given a good enough shake ... and if so, then we'll clear the form.

Let’s first look at the event handler.

window.addEventListener('devicemotion', update_dm, true);

Now let’s look at the update_dm() function

function update_dm(event){
   var acc_x = Math.abs(event.acceleration.x);
   var acc_y = Math.abs(event.acceleration.y);
   var acc_z = Math.abs(event.acceleration.z);
 	 if ( (acc_x || acc_y || acc_z) > 9.5){
 	      name.value = "";
 	      email.value = "";
 	      age.value = "";
 }

So there you have it. See the final demo of web forms which clear on shake and look through the code. Once again, make sure to view this in a browser which supports device orientation, like our experimental build of Opera Mobile for Android.