Think Different: jQuery webcam plugin

The jQuery webcam plugin is a transparent layer to communicate with a camera directly in JavaScript.

Overview

This plugin provides three different modes to access a webcam through a small API directly with JavaScript - or more precisely jQuery. Thus, it is possible to bring the image on a Canvas (callback mode), to store the image on the server (save mode) and to stream the live image of the Flash element on a Canvas (stream mode). If you just want to download the plugin, click here:

Download the jQuery webcam plugin

jQuery webcam example

jQuery

Take a picture after 3 seconds | Take a picture instantly

Available Cameras

HP Webcam [2 MP Fixed] (V4L2)

If you activate the filter with the button on the right side of the picture, methods of my already published jQuery plugin xcolor will be used to distort the colors of the Canvas.

General information about the interface

The following snippet describes the interface of the webcam API:

$("#camera").webcam({
 width: 320,
 height: 240,
 mode: "callback",
 swffile: "/download/jscam_canvas_only.swf",
 onTick: function() {},
 onSave: function() {},
 onCapture: function() {},
 debug: function() {},
 onLoad: function() {}
});

Config Parameter

width
The width of the flash movie.
height
The height of the flash movie. Both parameters have to be changed in the Flash file as well. Follow the instructions below to recompile the swf after the size change.
mode
The storage mode can be one of the following: callback, save, stream. Details about the usage of each parameter can be found under the according heading below.
swffile
Points to the swf file of the Flash movie, which provides the webcam API. There are two swf files provided via the download archive: jscam.swf, which provides the full API and jscam_canvas_only.swf which have no embedded JPEG library (I embedded an adjusted JPGEncoder of the AS 3 corelib). Thereby, the file is only one third as large as the original.
onTick, onSave, onCapture
These callbacks are described in detail below, since they change with each mode.
onLoad
The onLoad callback is called as soon as the registration of the interface is done. In the example above, I use the callback to get a list of all cameras available:

onLoad: function() {

    var cams = webcam.getCameraList();
    for(var i in cams) {
        jQuery("#cams").append("
"
 + cams[i] + "

"); } } Once the onLoad callback is called, a global object window.webcam is available, which provides the following methods:

capture([delay])
Captures an image internally.
save([file])
Saves the captured image accordingly to the storage mode.
getCameraList()
Get's an array of available cameras. If no camera is installed, an error is thrown and an empty array is returned.
setCamera([index])
Switches to a different camera. The parameter is the index of the element in the resulting array of getCameraList()

debug
The debug callback is called whenever there is a note or an error you should be notified. In the example above, I just replace the html content of the output container:

debug: function (type, string) {
 $("#status").html(type + ": " + string);
}

Callback Interface

The callback mode is used to get the raw data via a callback method to write it on a canvas element for example. The example above uses the callback mode.
As for the processing, one can imagine how it works as follows: Once the user has completely loaded the entire page and has accepted the security setting of Flash, she should be able to see herself. Then, the user triggers the method window.capture(). This may optionally receive a parameter that specifies the time to wait until the image is shot. To view the passage of time, the method onTick() is called after every second. The received parameter of this method is the amount of seconds remaining. In the example above, I simply change the status message like this:

onTick: function(remain) {

    if (0 == remain) {
        jQuery("#status").text("Cheese!");
    } else {
        jQuery("#status").text(remain + " seconds remaining...");
    }
}

Is copying finished, the onCapture callback is called, which in the example of above immediately calls the method webcam.save() to ultimately write the image to the canvas. The sample code also contains a small gimmick to simulate a flash using a lightbox and jQuery's fadeOut() fx method.

onCapture: function () {

 jQuery("#flash").css("display", "block");
 jQuery("#flash").fadeOut("fast", function () {
  jQuery("#flash").css("opacity", 1);
 });

 webcam.save();
}

In callback mode, for every line the callback onSave() is invoked, which gets an integer CSV of color values (separator is the semicolon). To write the data on the canvas, I use the following method in the example above:

onSave: function(data) {

    var col = data.split(";");
    var img = image;

    for(var i = 0; i < 320; i++) {
        var tmp = parseInt(col[i]);
        img.data[pos + 0] = (tmp >> 16) & 0xff;
        img.data[pos + 1] = (tmp >> 8) & 0xff;
        img.data[pos + 2] = tmp & 0xff;
        img.data[pos + 3] = 0xff;
        pos+= 4;
    }

    if (pos >= 4 * 320 * 240) {
        ctx.putImageData(img, 0, 0);
        pos = 0;
    }
}

Save Interface

From the view of processing, the save mode is almost identical to the callback mode. The only difference is that the webcam.save() method get's the file name passed as parameter. Then the shot photo is sent via HTTP_RAW_POST_DATA to the server and can be read for example with the following snippet to store or further process it in any way (Warning, input validation is not considered here!).

webcam.save('/upload.php');

And on the server side, you get the image like this:

php

$str = file_get_contents("php://input");
file_put_contents("/tmp/upload.jpg", pack("H*", $str));

?>

Alternative method to the upload via Flash

The Flash method has several problems. The implementation can lock the entire Flash movie and in the worst case the whole browser until the picture was uploaded sucessfully. A better approach is Ajax to upload the image asynchronously. Take a look at this example. It uploads a simple picture CSV if canvas elements are not implemented in the browser and sends a data url formatted string otherwise:

$(function() {

 var pos = 0, ctx = null, saveCB, image = [];

 var canvas = document.createElement("canvas");
 canvas.setAttribute('width', 320);
 canvas.setAttribute('height', 240);
 
 if (canvas.toDataURL) {

  ctx = canvas.getContext("2d");
  
  image = ctx.getImageData(0, 0, 320, 240);
 
  saveCB = function(data) {
   
   var col = data.split(";");
   var img = image;

   for(var i = 0; i < 320; i++) {
    var tmp = parseInt(col[i]);
    img.data[pos + 0] = (tmp >> 16) & 0xff;
    img.data[pos + 1] = (tmp >> 8) & 0xff;
    img.data[pos + 2] = tmp & 0xff;
    img.data[pos + 3] = 0xff;
    pos+= 4;
   }

   if (pos >= 4 * 320 * 240) {
    ctx.putImageData(img, 0, 0);
    $.post("/upload.php", {type: "data", image: canvas.toDataURL("image/png")});
    pos = 0;
   }
  };

 } else {

  saveCB = function(data) {
   image.push(data);
   
   pos+= 4 * 320;
   
   if (pos >= 4 * 320 * 240) {
    $.post("/upload.php", {type: "pixel", image: image.join('|')});
    pos = 0;
   }
  };
 }

 $("#webcam").webcam({

  width: 320,
  height: 240,
  mode: "callback",
  swffile: "/download/jscam_canvas_only.swf",

  onSave: saveCB,

  onCapture: function () {
   webcam.save();
  },

  debug: function (type, string) {
   console.log(type + ": " + string);
  }
 });

});

The server could then do something like this:

php

if ($_POST['type'] == "pixel") {
 // input is in format 1,2,3...|1,2,3...|...
 $im = imagecreatetruecolor(320, 240);

 foreach (explode("|", $_POST['image']) as $y => $csv) {
  foreach (explode(";", $csv) as $x => $color) {
   imagesetpixel($im, $x, $y, $color);
  }
 }
} else {
 // input is in format: data:image/png;base64,...
 $im = imagecreatefrompng($_POST['image']);
}

// do something with $im

?>

Stream interface

The stream mode is also quite the same procedure as the callback mode, with the difference that the onSave callback is called non-stop. The streaming starts with the method webcam.capture(). The webcam.save() method has no further effect.

Recompile the Flash binary

If you've made changes to the code or did just adjust the size of the video in the XML specification file, you can easily recompile the swf file from Linux console with the provided Makefile. You are required to install the two open source projects swfmill and mtasc that can be easily installed using apt-get under Debian/Ubuntu:

apt-get install swfmill mtasc
vim src/jscam.xml
make

Hint about empty screens after recompilation

There is a bug in the current version of swfmill. Please try to downgrade swfmill to 2.0.12, which fixes the issue!

Think Different

Tuesday

jQuery webcam plugin