Beej's Bit Bucket

 ⚡ Tech and Programming Fun

2010-02-09

HTML5's canvas Part II: Pixel Manipulation

Manipulated Pixels Your pixels will be manipulated.

In an earlier post, I wrote about a few of the things HTML5's <canvas> element could do. This post is about one more of them: direct pixel manipulation. You can basically do the three following things:

  1. Create an array of empty pixels, or
  2. Get an array of pixels from an existing canvas, or
  3. Set the pixels of a canvas from a pixel array

Once you have an array of pixels, you can manipulate it using the standard array accessors "[", "]". The pixels in the array are arranged in row-major order, and are represented as integers between 0 and 255, where each four-integer group represents the four color channels of one pixel: red, green, blue, and alpha (RGBA). What did I just say? Basically, it looks like this... say you have an image that's 3 pixels wide, and 2 pixels high. I know that's tiny, but I want to fit it all in the following super-busy diagram which shows the array indices for given elements of each pixel.

Pixel Layout Pixel layout in the pixel array for a 3-by-2 image of 6 pixels. Each pixel takes 4 elements in the array for red, green, blue, and alpha, for a total of 24 array elements, 0-23.

We'll get to a quick-and-easy calculation to find your way around the array in just a moment, but first, how does one create a pixel data array to be used on a canvas? You use the drawing context's createImageData() method. (Use getImageData() to get the array from an existing canvas.)

<canvas id="canvas1" width="100", height="100"></canvas>

<script type="text/javascript">
    element = document.getElementById("canvas1");
    c = element.getContext("2d");

    // read the width and height of the canvas
    width = element.width;
    height = element.height;

    // create a new batch of pixels with the same
    // dimensions as the image:
    imageData = c.createImageData(width, height);

It's going to be useful later to know the width of the image data, and fortunately that's baked into the ImageData object:

    imageData.width   // width of the image data in pixels
    imageData.height  // height of the image data in pixels
    imageData.data    // pixel data array of (width*height*4) elements</pre>

Now we have what we need to write a pixel-setting routine:

function setPixel(imageData, x, y, r, g, b, a) {
    index = (x + y * imageData.width) * 4;
    imageData.data[index + 0] = r;
    imageData.data[index + 1] = g;
    imageData.data[index + 2] = b;
    imageData.data[index + 3] = a;
}

There you can see how we first calculate the pixel "position" in the array (which is the row number y times the number of pixels in each row (the width), plus the column number x) and then we multiply it by 4 to get the first channel for that pixel, namely red, because each pixel takes up 4 array elements.

Eat your heart out, Jackson Pollock.

Exercise: write a getPixel(x,y) function that returns the RGBA values for a given pixel as an array, or as a packed 32-bit number (0x_RRGGBBAA_).

We've seen how to create an image pixel data array, and we've seen how to access it. The only thing left to do is to "paste" the pixels back onto the canvas so that we see them. We do this with a call to putImageData(), to which we pass the image data object, plus an X and Y offset at which to stamp the new data. Let's use this to produce a canvas that is covered with randomly-colored pixels:

<canvas id="canvas1" width="100", height="100">
Random Canvas
</canvas>

<script type="text/javascript">
function setPixel(imageData, x, y, r, g, b, a) {
    index = (x + y * imageData.width) * 4;
    imageData.data[index+0] = r;
    imageData.data[index+1] = g;
    imageData.data[index+2] = b;
    imageData.data[index+3] = a;
}

element = document.getElementById("canvas1");
c = element.getContext("2d");

// read the width and height of the canvas
width = element.width;
height = element.height;

// create a new pixel array
imageData = c.createImageData(width, height);

// draw random dots
for (i = 0; i < 10000; i++) {
    x = Math.random() * width | 0; // |0 to truncate to Int32
    y = Math.random() * height | 0;
    r = Math.random() * 256 | 0;
    g = Math.random() * 256 | 0;
    b = Math.random() * 256 | 0;
    setPixel(imageData, x, y, r, g, b, 255); // 255 opaque
}

// copy the image data back onto the canvas
c.putImageData(imageData, 0, 0); // at coords 0,0

</script>

But that's so ugly. Let's do something a little prettier, by setting the color of each pixel depending on the sine of the distance it is away from some point on the canvas:

pos = 0; // index position into imagedata array

xoff = width / 3; // offsets to "center"
yoff = height / 3;

// walk left-to-right, top-to-bottom; it's the
// same as the ordering in the imagedata array:

for (y = 0; y < height; y++) {
	for (x = 0; x < width; x++) {
		// calculate sine based on distance
		x2 = x - xoff;
		y2 = y - yoff;
		d = Math.sqrt(x2*x2 + y2*y2);
		t = Math.sin(d/6.0);

		// calculate RGB values based on sine
		r = t * 200;
		g = 125 + t * 80;
		b = 235 + t * 20;

		// set red, green, blue, and alpha:
		imageData.data[pos++] = Math.max(0,Math.min(255, r));
		imageData.data[pos++] = Math.max(0,Math.min(255, g));
		imageData.data[pos++] = Math.max(0,Math.min(255, b));
		imageData.data[pos++] = 255; // opaque alpha
	}
}

And that'll give you one of these:

Yummy splashy TRONy goodness. Why use JavaScript to generate the image? How about because the JavaScript is only about 1400 bytes long?

Notice in that case how I didn't bother calling my setPixel() function—I just walked all the way through the array one element at a time (indexed by the pos variable.)

What's going on there with all that Math.max() and Math.min() stuff? That's clamping the value between 0 and 255, inclusive. The min() call makes sure the value is less than 255, and then the max() call makes sure that result is greater than 0. Every browser I tried would automatically clamp the values in the array between 0 and 255, but the HTML5 draft says thou shalt not use values outside that range, so you'd better do the clamping yourself.

Exercise: write a script that generates a gradient that goes from black on the top to yellow on the bottom. Then modify it to also make the gradient go from black on the left to blue on the right.

There's just one final thing needed to wrap up this post, I feel, and that's to show manipulation of an existing canvas.

For this demo, we'll load an image, we'll draw that image onto the left half of the canvas (using the drawImage() method), then we'll get the pixel data (with the getImageData() method), and copy the image to the right half of the canvas—except along the way we'll be munging the colors to make it way more blue! Only about 87 million things get more exciting than that, yes?

Oh, goat! Why so blue?

There's a little bit of JavaScript code in here for loading the image, and the drawing all takes place in the image's onload callback, which is called when the image loading is complete.

<canvas id="cancan" width="400", height="200">Canvas Blue Goat</canvas>

<script type="text/javascript">
function imageLoaded(ev) {
    element = document.getElementById("cancan");
    c = element.getContext("2d");

    im = ev.target; // the image, assumed to be 200x200

    // read the width and height of the canvas
    width = element.width;
    height = element.height;

    // stamp the image on the left of the canvas:
    c.drawImage(im, 0, 0);

    // get all canvas pixel data
    imageData = c.getImageData(0, 0, width, height);

    w2 = width / 2;

    // run through the image, increasing blue, but filtering
    // down red and green:

    for (y = 0; y < height; y++) {
        inpos = y * width * 4; // *4 for 4 ints per pixel
        outpos = inpos + w2 * 4
        for (x = 0; x < w2; x++) {
            r = imageData.data[inpos++] / 3; // less red
            g = imageData.data[inpos++] / 3; // less green
            b = imageData.data[inpos++] * 5; // MORE BLUE
            a = imageData.data[inpos++];     // same alpha

            b = Math.min(255, b); // clamp to [0..255]

            imageData.data[outpos++] = r;
            imageData.data[outpos++] = g;
            imageData.data[outpos++] = b;
            imageData.data[outpos++] = a;
        }
    }

    // put pixel data on canvas
    c.putImageData(imageData, 0, 0);
}

im = new Image();
im.onload = imageLoaded;
im.src = "goat200.jpg"; // code assumes this image is 200x200

</script>

Exercise: modify the goat blueifier to instead perform Sobel edge detection on the image. (Follow this link for a tutorial and C code.)

Have fun!

References:

See Also:

Historic Comments

 David 2010-03-09 11:19:06

I was just curious if using the putImageData() API method was faster for animation purposes?

 beej 2010-03-09 19:09:22

@David It would depend on the animation, I think. For instance, if you use the CSS transform property to rotate a bitmap and put it on the screen in the DOM, I would think that had a chance of being faster than a putImageData() having done the rotation yourself. (That is, the path to the screen for the bitmap in the DOM might have been a step shorter.)

But I've seen examples with putImageData() that were absolutely fast enough.

One trick you can use is to pass the "dirty" parameters to putImageData(), so that only the updated part of the data is actually drawn.

Another trick (if you're drawing the same data over and over) is to draw your data to a small canvas, and then use drawImage() to stamp it on the display canvas. Again, my gut feeling is this would be faster than putPixelData() for the same usage, but I'm not really sure; it might be implementation-dependent, too.

 Kevin 2010-03-11 00:37:58

I wrote a Mandelbrot and Julia set viewer using this technique.
You can see it here:
http://www.scale18.com/canvas2.html

 beej 2010-03-11 01:04:52

@Kevin Cool! I put a link to it on my Mandelbrot Set article.

 Filippo Gregoretti 2010-06-26 17:57:30

Thanks for the post.
Canvas is so disappointing... I cannot even rotate a single bitmap without extremely complex and processor intensive calculations... How can I ever build a seriously engaging user experience with it?

 beej 2010-07-04 00:52:24

@Filippo Gregoretti Ah, but you can rotate a single bitmap very very quickly with canvas.

You'll want to set the rotation in the drawing context (context.rotate()), then drawImage() the image on the canvas. This uses the fast internal blitting to make it happen in a hurry.

So you can do 2D transformations very quickly with canvas, and even some kinds of compositing very quickly. Other things, like conditionally manipulating each pixel, you will have to do the slow way.

 Vidar 2010-07-17 00:36:34

I'm making a javascript/html5/php/mysql online 2d strategy game. It will be posted at cyberempire.net. game site Don't mind the java stuff there, it was the plan to make it in java, but I thought I would try with javascript. My problem before was finding an easy way to rotate images, problem solved with canvas. But, I would appreciate it if anybody had a fast collision detection routine.

 beej 2010-07-17 04:16:47

@Vidar If you're going HTML5, you also have the option to rotate individual DOM elements (like images) in CSS3 with the "transform" property. (But of course, using canvas is an option, as well.)

As for collision detection, check out this fantastic page at Gamasutra:

http://www.gamasutra.com/view/feature/3383/simple_intersection_tests_for_games.php

Page 5 talks about "Oriented Bounded Boxes" which might be along the lines of what you need.

 Micah Smith 2011-02-10 15:03:54

Great article, and great writing style.

 OrNot 2011-07-17 12:17:57

Nice article.

I have a question:

I modify your code as below:


c.translate(50,50);

// put pixel data on canvas
c.putImageData(imageData, 0, 0);


Why is the output image still in (0,0) not (50,50)?

 michael 2011-09-01 04:35:03

I too have the same question as OrNot, although what I want to do is to read the pixel at (x,y) after a translate and rotation. Is this possible? perhaps there is a way to get the "real" coordinates of x and y after a translate and rotate has been done, and get the pixel at the "real" coordinates?

 beej 2011-10-14 16:35:26

@OrNot getImageData() and putImageData() operate on a low level, referring to the pixels in the bitmap backing store, and are not affected by the drawing context.

In the spec: "The current path, transformation matrix, shadow attributes, global alpha, the clipping region, and global composition operator must not affect the getImageData() and putImageData() methods."

 beej 2011-10-14 16:42:17

@michael This is tricky for a couple reasons. (If it's possible, given your needs, to read the pixel at x,y before the transform, it could be a lot better.)

First of all, the math isn't hard, and there are a number of JS libraries to do it. (I've used Sylvester: http://sylvester.jcoglan.com/ ). Basically, you have to run the math again in parallel to get the result.

However, canvas will render to a subpixel. This means that part of your color could be on one pixel, and the rest would be on another. It might be a while pixel, originally, but it might turn to several gray pixels after rotation.

 brock 2011-11-10 17:29:27

Is it possible to get the raw pixel value of an 8-bit indexed png, not the associated RGBA? I have images where the values (0-255) in the raw pixels have an actual data meaning. Depending on the time of year, the actual index palette changes, (so the displayed colors change) but not the 0-255 values in each pixel. I can do this client side, but with overhead which I don't want.

What I want to do is a browser side calculation. So when a user mouses over an image, the x,y provides the pixel needed, and the 0-255 is extracted. This is then converted to a data value (using a know formula,) which can be displayed for the user (and of course, a coresponding lat long which can be determined since I know the outer lat/long bounds of the image)

 brock 2011-11-10 18:24:15

@brock
I meant "I can do it server side,..."

 beej 2011-12-02 19:21:23

@brock I'm unaware of a way to get this low level data from the PNG, short of writing a PNG decoder in JS, and even then, my JS-fu isn't strong enough to know a way to get the binary data in JS.

By the time the data gets exposed, it's already abstracted. :-/

 dave 2012-03-21 22:02:58

Hi Brock, were you able to find a way to get the raw PNG values in JS? I have a similar requirement and want to use IMG if possible. Thanks!
-dave

Comments

Blog  ⚡  Email beej@beej.us  ⚡  Home page