Easy HTML5 canvas and CSS Sprite texture atlas

Texture atlas is a collection of images, all composed in single large image. The game or HTML5 app can load this single image instead of wasting time for requesting every small image separately.

I wrote tool that can be used to package collection of images to single atlas and let web app use that conveniently both as CSS sprite or in HTML5 canvas drawing.

Tool itself is Python script that uses ImageMagick commands to read and write images. The output is the atlas image, CSS, raw JSON and Javascript import file. CSS file defines CSS sprite for each image. JSON is raw data of each images position in the atlas, Javascript import file can be included on HTML page and it loads the atlas position info into global variable.

Setup

Get script from here: https://raw.github.com/tikonen/blog/master/packer/packer.py

Make it executable

$ chmod +x packer.py

Verify that all is ok

~/work/blog/packer $ ./packer.py -h
usage: packer.py [-h] [-o OUTFILE] [-jo JSONOUTFILE] [-jso JSOUTFILE]
                 [-co CSSOUTFILE] [-p PAD] [-mw WIDTH] [-mh HEIGHT]
                 FILE [FILE ...]

Packs images to atlas. Uses ImageMagick to parse and compose the images

positional arguments:
  FILE             Image file

optional arguments:
  -h, --help       show this help message and exit
  -o OUTFILE       Output atlas file
  -jo JSONOUTFILE  Output atlas json file
  -jso JSOUTFILE   Output atlas import js file
  -co CSSOUTFILE   Output atlas css file
  -p PAD           Padding
  -mw WIDTH        Maximum width
  -mh HEIGHT       Maximum height

Install ImageMagick easily on OS/X with Macports or Homebrew.

$ sudo port install ImageMagick

Verify that ImageMagick installation works

$ identify --version
Version: ImageMagick 6.8.7-3 2013-10-28 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2013 ImageMagick Studio LLC
Features: DPC
Delegates: bzlib djvu fftw fontconfig freetype gslib jng jpeg lcms ltdl lzma png ps png tiff webp x xml zlib

Usage

You use script by giving image file as arguments and defining the desired output files and maximum image dimensions

For example:

$ ./packer.py img/*.png -o sprites.png

This would produce sprites.png, sprites.json, sprites.css and sprites.json.js

Options
Following options are supported

  • -p PAD
    Defines padding. The amount of empty pixels around each image. This prevents scaling artifacts on HTML canvas use when drawing from decimal source coordinates.
  • -mw WIDTH,-mh HEIGHT
    Maximum width and height of output image.
  • -o FILE
    Output image file.
  • -jo FILE, -jso FILE, -co FILE
    Output JS import, JSON and CSS file.

Example

We have following small images that are needed in our web app. Loading them all independently would take 5 HTTP requests.

  • button_minus.png
  • button_plus.png
  • cannon_marker.png
  • pause.png
  • status_bar.png

montage

Running tool converts them to single atlas

~/work/blog/packer $ ./packer.py example/pics/* -o example/html/sprites.png -mw 512
Checking ImageMagick
Found: Version: ImageMagick 6.8.7-3 2013-10-28 Q16 http://www.imagemagick.org
===========================
Resolving file dimensions
button_minus.png -> 53x43
button_plus.png -> 53x42
cannon_marker.png -> 43x28
pause.png -> 53x42
status_bar.png -> 496x74
===========================
fitting 5 images, padding 1
successfully fitted 5 images to 496x118 padding 1
Wrote: atlas to example/html/sprites.png
Wrote json to example/html/sprites.json
Wrote js to example/html/sprites.json.js
Wrote css to example/html/sprites.css

In this example we defined maximum width of 512 pixels. The resulting image is cropped to minimum size 496×118.

sprites
This atlas can be used now in two different ways on the web page.

CSS Sprite

Web page includes the generated CSS file and defines class by filename (e.g. “bg-sprites status_bar” for each element that uses sprite.

<!DOCTYPE html>
<html>
<head>
    <title>Example</title>
    <link rel="stylesheet" type="text/css" href="sprites.css">
    <style type="text/css">
        .bg-sprites {
            color: white;
            text-align: center;
        }
    </style>
</head>
<body>
    <div>
        <h1>Example</h1>
        <div>
            <h2>Cannon - simple sprite with image</h2>
            <div class="bg-sprites cannon_marker">
            </div>
            <h2>Status bar - sprite with text inside</h2>
            <div class="bg-sprites status_bar">
                <span style="font-size:xx-large;position:relative;top:15px;">Here is some text</span>
            </div>
            <h2>Buttons - list</h2>
            <div>
                <span style="float:left;margin:5px;" class="bg-sprites button_minus"></span>
                <span style="float:left;margin:5px;" class="bg-sprites button_plus"></span>
                <span style="float:left;margin:5px;" class="bg-sprites pause"></span>
            </div>
        </div>
    </div>
</body>

This renders following page:
example css sprite

See example page here: http://ikonen.me/examples/packer/example_css.html

HTML5 Canvas

Another option is to use it with HTML canvas. Page links the generated JSON import script and loads the image.

<!DOCTYPE html>
<html>
<head>
    <title>Example</title>
    <style type="text/css">
        canvas {
            width: 600px;
            height: 400px;
        }
    </style>
</head>
<body>
    <div>
        <h1>Canvas Example</h1>
        <div>
            <canvas id="thecanvas" width="600" height="400"></canvas>
        </div>
    </div>
  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js" ></script>
  <script type="text/javascript" src="sprites.json.js"></script>
  <script type="text/javascript">
      $(function() {
          var canvas = document.getElementById('thecanvas');
          var ctx = canvas.getContext('2d');
          ctx.strokeStyle = 'hotpink';
          ctx.strokeRect(0, 0, 600, 400);
          ctx.font = '20px Arial';

          // Load image and the json that defines locations
          var sprites = new Image();
          sprites.src = 'sprites.png';
          sprites.addEventListener("load", function() {
                  var assets = bg_sprites; // imported by sprites.json.js

                  // draw them all
                  var xoffset = 5,
                      yoffset = 25;

                  for (var pic in assets) {
                      var asset = assets[pic];
                      ctx.fillText(pic, xoffset, yoffset-3);

                      // draw image from sprite
                      ctx.drawImage(sprites, asset.x, asset.y, asset.w, asset.h, xoffset, yoffset, asset.w, asset.h);

                      yoffset += asset.h + 20;
                  }
          }, false);
      });
  </script>
</body>
</html>

This renders following page:

example canvas

See example page here: http://ikonen.me/examples/packer/example_canvas.html

Get complete example from Github.

Reduce mp3 file size for app and web use

Typical 2 minutes piece of music is roughly 2-4MB in common mp3 format. This is size for usual quality that is stereo audio 256 kbps and 44.1 kHz sample rate. Tough the file size is not that large, it will slow down loading HTML5 game or mobile app, especially on wireless networks.

Web HTML5 or mobile game applications do not need nearly as high quality and the size can be reduced to ~400-700kB without too much audible quality penalty. Difference can be noticed if you listen for it, but it’s unlikely that users do notice anything, as they have nothing to compare it against. This also depends lot of the music, sharp high frequency sounds tend to distort first.

MP3/WAV to reduced size MP3

Easy way to convert mp3s is lame. A flexible command line mp3 decoder and encoder.

Here we use 1:45 minutes 1.7MB music piece Lake Hylia from Zelda Reorchestrated. I don’t own any rights to this music and it’s used purely for demonstration purposes. I selected this piece because it’s melodic and instrumental like most game music is.

Key to reducing file size is lower sample and bitrate, and most importantly converting file to mono. Example below is how to do this with lame 3.99.5 on OS/X. I installed lame using Macports.

$ lame -hv -mm --resample 22.05 "Zelda Reorchestrated - Lake Hylia.original.mp3" -B 32 "Zelda Reorchestrated - Lake Hylia.small.mp3"
LAME 3.99.5 64bits (http://lame.sf.net)
Autoconverting from stereo to mono. Setting encoding to mono mode.
Resampling:  input 44.1 kHz  output 22.05 kHz
polyphase lowpass filter disabled
Encoding Zelda Reorchestrated - Lake Hylia.mp3
      to Zelda Reorchestrated - Lake Hylia.small.mp3
Encoding as 22.05 kHz single-ch MPEG-2 Layer III VBR(q=4)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  3996/3996  (100%)|    0:01/    0:01|    0:01/    0:01|   82.375x|    0:00
  8 [  36] **
 16 [  24] *
 24 [   4] *
 32 [3932] **************************************************************************************************************************************
-------------------------------------------------------------------------------------------------------------------------------------------------
   kbps       mono %     long switch short %
   31.7      100.0        99.9   0.1   0.1
Writing LAME Tag...done
ReplayGain: +3.7dB

This converts the input mp3 (lame also accepts .wav files) to 22.05kHz 64kbs monoaural mp3. The output file size is 412kB, resulting to nearly 75% savings. You can reduce file further by using even lower bitrate, (experiment -B 24 or -B 16), but you may start hearing distortions.

$ du -hs Zelda\ Reorchestrated\ -\ Lake\ Hylia.*
1.6M	Zelda Reorchestrated - Lake Hylia.original.mp3
404K	Zelda Reorchestrated - Lake Hylia.small.mp3

Test play the files here and listen for any differences.

Original: Zelda Reorchestrated – Lake Hylia.original.mp3
Reduced: Zelda Reorchestrated – Lake Hylia.small.mp3

MP3/WAV to reduced OGG Vorbis

This is main for Firefox as it still supports only Ogg Vorbis. You can convert mp3/wav file to reduced mono ogg vorbis with ffmpeg tool.

This example uses ffmpeg 2.1, and like lame I installed is on OS/X using Macports.

$ ffmpeg -i "Zelda Reorchestrated - Lake Hylia.original.mp3" -strict experimental -acodec libvorbis -ab 32k -ar 22050 -ac 1 "Zelda Reorchestrated - Lake Hylia.small.ogg"
ffmpeg version 2.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Oct 30 2013 23:24:10 with Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass --enable-libbluray --enable-gnutls --enable-libfreetype --disable-outdev=xv --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-nonfree --enable-libfaac
  libavutil      52. 48.100 / 52. 48.100
  libavcodec     55. 39.100 / 55. 39.100
  libavformat    55. 19.104 / 55. 19.104
  libavdevice    55.  5.100 / 55.  5.100
  libavfilter     3. 90.100 /  3. 90.100
  libavresample   1.  1.  0 /  1.  1.  0
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mp3, from 'Zelda Reorchestrated - Lake Hylia.original.mp3':
  Metadata:
    title           : Lake Hylia
    artist          : Zelda Reorchestrated
    album           : Twilight Princess
    track           : 11
  Duration: 00:01:44.39, start: 0.000000, bitrate: 128 kb/s
    Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s
Output #0, ogg, to 'Zelda Reorchestrated - Lake Hylia.small.ogg':
  Metadata:
    title           : Lake Hylia
    artist          : Zelda Reorchestrated
    album           : Twilight Princess
    track           : 11
    encoder         : Lavf55.19.104
    Stream #0:0: Audio: vorbis (libvorbis), 22050 Hz, mono, fltp, 32 kb/s
    Metadata:
      title           : Lake Hylia
      artist          : Zelda Reorchestrated
      album           : Twilight Princess
      TRACKNUMBER     : 11
      encoder         : Lavf55.19.104
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 -> libvorbis)
Press [q] to stop, [?] for help
size=     398kB time=00:01:44.38 bitrate=  31.2kbits/s
video:0kB audio:387kB subtitle:0 global headers:3kB muxing overhead 1.866299%

Resulting OGG file size and quality is comparable to reduced mp3.

$ du -hs Zelda\ Reorchestrated\ -\ Lake\ Hylia.*p3
400K	Zelda Reorchestrated - Lake Hylia.small.ogg
1.6M	Zelda Reorchestrated - Lake Hylia.original.mp3

Compare here:

Original: Zelda Reorchestrated – Lake Hylia.original.mp3
Reduced: Zelda Reorchestrated – Lake Hylia.small.ogg

Simple Slot machine game using HTML5 Part 4: Offline mode

This is the fourth part of the Slot machine game in HTML5 (previous parts 1, 2, and 3) and this time we modify the game to support HTML5 offline mode, also known as HTML5 Application Cache.

Slots Offline

Try out offline supported version here.

Word of warning. HTML5 offline mode is powerful but very fragile feature. It’s tricky to get right, but once you get it to work the mobile user experience can be very native app like.

Some problems you will encounter

  • Browser refresh logic is confusing. Especially the fact that browser does not use updated manifest and resources when they change but only after next reload. Fortunately Javascript workarounds exists.
  • Application cache file maintenance needs diligence. For example, browser will not reload any assets if this file is not modified.
  • Externally linked resources do not generally work offline. This makes CDN use difficult.
  • Web server has to use right MIME type and cache settings to reliably use application cache files. Most web servers don’t do this in default configuration.
  • No reliable way to detect if page was loaded in online or offline mode.
  • Chrome bypasses some restrictions (e.g. cross-domain issues) in the specification and what works in Chrome may not work anywhere else.
  • Each browser has slightly different meaning and heuristic for using offline mode. For example if browser can load some unrelated pages but can’t currently load your app page it may not show your page in offline mode and simply shows “Can not reach the server error”. This may happen especially if it knows from last load that manifest has been updated. Then at other times, it may load page few times in offline mode even when connectivity has returned.

Manifest file

Web page must use application cache manifest file to support offline mode. This manifest file is specified in html tag of the page.

<!DOCTYPE html>
<html manifest="slots.appcache">
<head>
 ...

The file has listing of all content that page needs. For detailed explanation of each section, refer to Beginners guide to HTML5 application cache

CACHE MANIFEST
# version 10

NETWORK:
# here goes resources that must be never cached
js/online.json

CACHE:
audio/nowin.mp3
audio/nowin.ogg
audio/roll.mp3
audio/roll.ogg
audio/slot.mp3
audio/slot.ogg
audio/win.mp3
audio/win.ogg
css/webfonts.css
css/reset.css
css/slot.css
css/Slackey.ttf
img/build-64.png
img/cash-64.png
img/energy-64.png
img/gold-64.png
img/goods-64.png
img/loading.gif
img/staff-64.png
js/slot.js
js/jquery.min.js
index.html

Web Fonts

Web fonts must be hosted locally if you want to use them offline. Note that some web fonts may have licensing restrictions for local hosting.

<link type='text/css' rel='stylesheet' href="css/webfonts.css"/>

The webfont.css defines the font face and loads true type file.

@font-face {
  font-family: 'Slackey';
  font-style: normal;
  font-weight: 400;
  src: local('Slackey'), url(Slackey.ttf) format('truetype');
}

The file Slackey.ttf is hosted locally in css directory.

Web Server Support

Web server must use correct MIME type for text/cache-manifest application cache manifest. For example, in NGINX web server edit the mime.types and add following file type to MIME type mapping.

text/cache-manifest		appcache;

Browsers should check manifest every time page is loaded online, but it may not do this often enough if cache control is too long. Therefore, set short cache lifetime for the manifest files by adding this inside server section of NGINX configuration file. This forces cache lifetime of 1 minute to all *.appcache files.

# set 1 minute cache life for HTML5 offline manifests
location ~* \.(appcache)$ {
   expires 1m;
}

Verify with cURL that server response Content-Type has right MIME type and that the Expires and/or Cache-Control have correct 1 minute cache life time. If you get 404 error, make sure that site root configuration is set in http section.

$ curl -I http://localhost:8081/slots.appcache
HTTP/1.1 200 OK
Server: nginx/1.4.0
Date: Sun, 05 May 2013 04:43:32 GMT
Content-Type: text/cache-manifest
Content-Length: 38
Last-Modified: Sun, 05 May 2013 04:25:28 GMT
Connection: keep-alive
ETag: "5185df38-26"
Expires: Sun, 05 May 2013 04:44:32 GMT
Cache-Control: max-age=60
Accept-Ranges: bytes

Detecting online status

Currently only “reliable” method to do is to make Ajax request and check the response. There are some caveats

  • Request may fail for other reasons, and this does not mean browser is in offline mode
  • Offline status may change while user is in page, you may want to do repeat polling check.
  • User may be in public WiFi that redirects requests to login server. This can confuse your app that gets response but is not what was expected.

Slots game checks online status in parallel while game loads and only on startup. Slots game does not really need to know if it’s online or offline, but just writes the status on screen for debugging purposes.

<script type="text/javascript">$(function () {

    var game = SlotGame();

    // Attempt loading static json file from server to detect online or offline mode.
    // The url has unique random parameter to avoid browser or proxy caches
    $.ajax({
        url: 'js/online.json?ts=' + (~~new Date()),
        dataType: 'json',
        success: function(data) { 
            if ( data.online ) { 
                game.setOnlineStatus(true);
            } else {
                // might be online, but we didn't get expected response. Could be
                // e.g. Wifi login page.
                game.setOnlineStatus(false);
            }
        },
        error: function() {
            game.setOnlineStatus(false);
        }
    });

});</script>

Otherwise loading images, audio and other content should be fully transparent to your app. Think twice before doing separate logic for online and offline as things will get difficult. Best advice I can give is to that you write code for online use with proper handling for Ajax errors. In this way when app loads in online mode but loses network later in session (e.g. when user goes in subway tunnel), the experience does not break completely.

Testing

This is the part where things get interesting, offline is tricky to test because of caching and browser reload logic. See detailed lamentation about subject here in Dive into HTML5.

These are the best practices I’ve come up with. First, set browser manually to offline mode to try things out. e.g. in Firefox this is enabled from File->Work Offline.

Firefox offline

Second, if you develop the game from local server, do not use http://localhost as host, but use real domain name that resolves to localhost. In this example I’ve used http://hexxie.com that supports wildcard subdomain. Any subdomain resolves to address 127.0.0.1.

$ nslookup anything-goes-here.hexxie.com
Server:		192.168.1.254
Address:	192.168.1.254#53

Non-authoritative answer:
Name:	anything-goes-here.hexxie.com
Address: 127.0.0.1

In this way you can always start from scratch the offline debugging simply by changing subdomain name. For example I just used slotsoff1.hexxie.com, slotsoff2.hexxie.com, … etc.:

Trick url

Note that at least Firefox asks each time if you allow offline content.
Accept offline

Before each deploy, remember to increment the version comment in manifest file, so web server notices that the file has changed and browser will refresh it on next load. Server does not look inside the manifest file, so it does not matter how you change the file, as long as it’s changed.

Good Luck!

Code is available in Github.

Simple Slot machine game using HTML5 Part 3: Loading

UPDATE: See also Simple Slot machine game using HTML5 Part 4: Offline mode

In previous part 1 and part 2 I implemented simple slots machine purely in HTML5 with audio support.
After adding audio, the initial loading time increased a lot up to several seconds. This slowdown is easy to miss during development, as developer has always primed browser cache and content is loaded from development server that is hosted locally or even in developers machine.
It’s very important to occasionally clear the browser cache and put the game to remote server and reload the game to get realistic estimation of new users loading time (i.e. the empty cache experience).

This part improves the landing experience by adding loading progress bar so users know that the game is loading and will start soon.

Loading screen

Try out version with loading bar here.

How it works

Loading bar is simple nested div, where parent draws the outline and the child is the filling progress bar.

<div id="progressbar">
   <div id="progress"></div>
</div>

The CSS rules set the colors and position.

#progressbar
{
    margin-top: 10px;
    background: black;
    border: 1px solid mediumaquamarine;
    width: 80%;
    height: 15px;
    margin-left: auto;
    margin-right: auto;
}
#progressbar #progress
{
    height: 100%;
    position: relative;
    width: 0%;
    left: 0;
    background: #77e0fb;
}

Indicating progress is now simply matter of updating the width of #progress div.

Any HTML5 game loading bar implementation should be done purely with HTML and CSS and not use any images, as this just makes total loading experience slower and loading bar does not appear immediately when page renders. If you must use images, consider using base64 embedded images.

#progressbar {
  background-image:url(data:image/png;base64,iAE28w0KGgoABBANSAxhEegBB54DIA...);
  ...
}

If possible, void building the loading view with javascript as your page must load javascript before being able to show anything.
For best experience you should always have the CSS file in the <head> section of the HTML file and the javascript at the end of file, just before closing </body>. This way page can render with proper layout before any javascript has been loaded.

<!DOCTYPE HTML>
<html>
<head>
    ...
    <link href='http://fonts.googleapis.com/css?family=Slackey' rel='stylesheet' type='text/css'/>
    <link type="text/css" rel="stylesheet" href="css/reset.css"/>
    <link type="text/css" rel="stylesheet" href="css/slot.css"/>

</head>
<body>
<div id="viewport">
    ...
</div>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js"></script>
<script type="text/javascript" src="js/slot.js"></script>
<script type="text/javascript">$(function () { SlotGame(); });</script>
</body>
</html>

Both image and audio assets are loaded with same preloader, that keeps track of the progress and updates the progressbar. When loading is completed it hides the loading view. See the code for details how the preloader is used to load images and web/html5 audio files.

var progressCount = 0; // current progress count
var progressTotalCount = 0; // total count

function updateProgress( inc ) {
    progressCount += (inc || 1);
    if ( progressCount >= progressTotalCount ) {
        // done, complete progress bar and hide loading screen
        $('#progress').css('width', '100%');
        $('#loading').slideUp(600);
    } else {
        // Update progress bar
        $('#progress').css('width', parseInt( 100 * progressCount / progressTotalCount)  + '%');
    }
}

// Generic preloader handler, it calls preloadFunction for each item and
// passes function to it that it must call when done.
function preloader( items, preloadFunction, callback ) {

    var itemc = items.length;
    var loadc = 0;

    // called by preloadFunction to notify result
    function _check( err, id ) {
        updateProgress(1);
        if ( err ) {
            alert('Failed to load ' + id + ': ' + err);
        }
        loadc++;
        if ( itemc == loadc ) callback();
    }

    progressTotalCount += items.length;

    // queue each item for fetching
    items.forEach(function(item) {
        preloadFunction( item, _check);
    });
}

Improving loading time

You can further improve initial loading time by conventional web tricks

  • Enable HTTP gzip compression in the web server
  • Include jquery and library scripts from google api url, user may have those already in the cache
  • Merge and minify the javascript e.g. with uglify-js
  • Merge and minify the CSS with CSS compressor
  • Minify PNG image files with pngout. Use JPG where possible.
  • Compose all images in single montage and use that with CSS sprites and canvas ctx.drawImage
  • Convert audio files to mono to save half of the size. Use e.g. Audacity or ffmpeg
  • Host asset files in CDN, such as Akamai or Amazon S3. Note that if assets are loaded from different domain than the HTML document, you may encounter security problems if Cross-origin policy is not properly set for the content.

Use Chrome browsers audit tool get idea what takes most time.

Code is available in Github.

Continue to Simple Slot machine game using HTML5 Part 4: Offline mode

Simple Slot machine game using HTML5 Part 2: Audio

UPDATE: See also Simple Slot machine game using HTML5 Part 3: Loading.

In part 1 I presented simple slots machine purely in HTML5. This part extends the basic implementation with audio support. The game itself is simple slot machine that has only one winning line and we add effects for roll start, reel stop and win/loss.

Slots with audio

Try audio enabled game here. Note that loading time is significantly longer in audio enabled version. Debug text under button tells what audio system game uses for your browser.
Original non audio version from part 1 is here.

How to support Audio

Web game can implement audio in 3 main ways.

1. Flash player audio. (e.g. use SoundManager2 library)
2. HTML5 Audio (Buzz is easy way to use it)
3. Web Audio API. (See HTML5 Rocks for tutorial)

Flash audio is pretty much deprecated now, as only older browsers will still need it that most of don’t support HTML5 canvas anyway.
HTML5 Audio works very well for desktop browsers, but has only nominal support for mobile (Android & iOS). It’s generally too moody for tablets or mobile.
Web Audio API is supported only by latest browsers, but it works reliably e.g. in Safari iOS 6.0.

Web Audio API has two implementations in wild, the WebKit (iOS, Safari, Chrome) and the Standard (latest Firefox). Fortunately differences are small.

The libraries listed above simplify implementation a lot, but it’s easier to understand how these technologies work with simple examples. So I implemented both methods 2 and 3 from scratch.

Some caveats with audio.

  • Game initial loading time will increase. Audio files can be pretty large and they must be usually preloaded so they can be played on game start
  • iOS (iPad/iPhone) does not allow autoplay for audio. Audio must be enabled by playing some sound in click event handler.

Implementation

Initialization function accepts array of objects that have required audio file name in id property and callback that is called after audio has been initialized and loaded.
First code checks if mp3 or ogg is supported. Firefox requires .ogg and it’s easy to convert at least in OS/X or Linux with ffmpeg. Exact command line depends little on ffmpeg version.

$ ffmpeg -i win.mp3 -strict experimental -acodec vorbis -ac 2 win.ogg

When format is known, the code checks wether to use Web Audio API or normal HTML5 Audio.

function initAudio( audios, callback ) {

    var format = 'mp3';
    var elem = document.createElement('audio');
    if ( elem ) {
        // Check if we can play mp3, if not then fall back to ogg
        if( !elem.canPlayType( 'audio/mpeg;' ) && elem.canPlayType('audio/ogg;')) format = 'ogg';
    }

    var AudioContext = window.webkitAudioContext || window.mozAudioContext || window.MSAudioContext || window.AudioContext;

    if ( AudioContext ) {
        // Browser supports webaudio
        return _initWebAudio( AudioContext, format, audios, callback );
    } else if ( elem ) {
        // HTML5 Audio
        return _initHTML5Audio(format, audios, callback);
    } else {
        // audio not supported
        audios.forEach(function(item) {
            item.play = function() {}; // dummy play
        });
        callback();
    }
}

Both initialization functions attempt to load the desired format of needed audio files and sets play function in objects that is used to play the effect. If audio initialization fails, this play function is set to dummy empty function.

HTML5 Audio initialization function creates Audio objects and sets src to point to the audio file. Downloading is handled automagically by the browser.

function _initHTML5Audio( format, audios, callback ) {

    function _preload( asset ) {
        asset.audio = new Audio( 'audio/' + asset.id + '.' + format);
        asset.audio.preload = 'auto';
        asset.audio.addEventListener("loadeddata", function() {
            // Loaded ok, set play function in object and set default volume
            asset.play = function() {
                asset.audio.play();
            };
            asset.audio.volume = 0.6;
        }, false);

        asset.audio.addEventListener("error", function(err) {
            // Failed to load, set dummy play function
            asset.play = function() {}; // dummy
        }, false);

    }
    audios.forEach(function(asset) {
        _preload( asset );
    });
...

Web Audio initialization is bit more complicated, it needs to download the audio with XHR requests.

function _initWebAudio( AudioContext, format, audios, callback ) {

    var context = new AudioContext();

    function _preload( asset ) {
        var request = new XMLHttpRequest();
        request.open('GET',  'audio/' + asset.id + '.' + format, true);
        request.responseType = 'arraybuffer';

        request.onload = function() {
            context.decodeAudioData(request.response, function(buffer) {

                asset.play = function() {
                    var source = context.createBufferSource(); // creates a sound source
                    source.buffer = buffer;                    // tell the source which sound to play
                    source.connect(context.destination);       // connect the source to the context's destination (the speakers)
                    // support both webkitAudioContext or standard AudioContext
                    source.noteOn ? source.noteOn(0) : source.start(0);
                };
                // default volume
                // support both webkitAudioContext or standard AudioContext
                asset.gain = context.createGain ? context.createGain() : context.createGainNode();
                asset.gain.connect(context.destination);
                asset.gain.gain.value = 0.5;


            }, function(err) {
                asset.play = function() {};
            });
        };
        request.onerror = function(err) {
            asset.play = function() {};
        };
        request.send();
    }

    audios.forEach(function(asset) {
        _preload( asset );
    });

}

NOTE: Chrome supports XMLHttpRequest only when loading pages over HTTP. If you load the HTML file locally you’ll see erros like this in the error console:
XMLHttpRequest cannot load file:///Users/teemuikonen/work/blog/slot2/audio/roll.mp3. Cross origin requests are only supported for HTTP.

After audio has initialized, the game can play any effect simply by calling play function for the effect. If audio initialization or loading failed, the play is simply a dummy function.

$('#play').click(function(e) {
    // start game on play button click
    $('h1').text('Rolling!');
    game.audios[0].play(); // Play start audio
    ...

Code is available in Github.

Continue to Simple Slot machine game using HTML5 Part 3: Loading.

Minesweeper clone in HTML5

In my two previous block entries I wrote about one possible ways to do simple Slots and Car games on HTML5 technologies. This writeup combines some of those methods and introduces new ones to implement Minesweeper game with a twist.

mine

This game is actually “reverse” minesweeper, or should we say Applesweeper. Players mission is to find all the apples without clicking on of the empty tiles. Score is increased when player finds an apple and is decreased when he misses. Game ends when all the apples are found.

Try out the game here: http://ikonen.me/examples/mine/

As in previous games, hud is a html div, and the game grid is drawn on a canvas. Grey slab that hides tiles content is procedurally generated on offscreen canvas at the startup.

this.slab = document.createElement('canvas');
var ctx = this.slab.getContext('2d');
this.slab.width = this.resolution;
this.slab.height = this.resolution;

ctx.fillStyle = 'grey';
ctx.fillRect(0, 0, this.resolution, this.resolution);

ctx.beginPath();
ctx.fillStyle = 'white'
ctx.moveTo(0, 0);
ctx.lineTo(this.resolution, 0);
ctx.lineTo(this.resolution, this.resolution);
ctx.lineTo(0, 0);
ctx.closePath();
ctx.fill();

ctx.fillStyle = 'lightgrey';
ctx.fillRect(4, 4, this.resolution-8, this.resolution-8);

The this.slab holds off-screen canvas that contains the generated slab image.

Game area size and number of apples are function of screen size, to adapt to different screensizes.

var width  = window.innerWidth;
var height = window.innerHeight;

GRID_W = Math.min( 12, ~~(width / GRID_RESOLUTION));
GRID_H = Math.min( 12, ~~(height / GRID_RESOLUTION)) - 1;
var APPLE_COUNT = ~~((GRID_W * GRID_H) / 8);

For example, Here is the game on iPhone 3Gs.

mine_iphone

Game main loop is passive, when user clicks on the screen click handler sets the location on object that holds the clicked tile x and y coordinates

$('#container').click( function( e ){
    var p = $('#canvas').offset();        
    game.click = {
        x:parseInt((e.pageX-p.left)/game.resolution),
        y:parseInt((e.pageY-p.top)/game.resolution)};
    }
);

Main update handler is called on each frame and it checks if button has been clicked and updates the grid and redraws it if required. Grid is simple array where x and y are mapped as position.

Game.prototype.pos = function( x, y ) {
    return y*this.width+x;
}
Game.prototype.xy = function( pos ) {
    return {
        x:parseInt(pos%this.width),
        y:parseInt(pos/this.width)
    }
}

Grid array values are integers where content is bit masked. Higher bits are used to flag if grid location has slab and apple, empty or number.

var SLAB_MASK = Math.pow(2, 16);
var APPLE_MASK = Math.pow(2, 15);

// grid location (5, 6) has slab and number 3
var pos = this.pos(5, 6);
this.grid[pos] = 3;
this.grid[pos] |= SLAB_MASK;

The slab is removed from location just by negating it

this.grid[pos] &= ~SLAB_MASK;

Draw loop just checks for each position and checks with mask what it contains

for ( var y=0; y < this.height; y++ ) {
    for ( var x=0; x < this.width; x++ ) {
        // Draw each tile
        var s = this.pos(x,y);
       if (s & SLAB_MASK) {
          // Still covered tile
         this.ctx.drawImage( this.slab, x * this.resolution, y * this.resolution )

       } else if (s & APPLE_MASK) {
         // Uncovered apple
         this.ctx.drawImage( tile, x * this.resolution + 2, y * this.resolution + 2 )
       } else if (s > 0) {
         // Neighbour number
         this.ctx.fillText( '' + s ,
                            x * this.resolution + this.resolution/2,
                            y * this.resolution + this.resolution/2)
       }
    }
}

When player clicks on empty tile, recursive function walks through the grid clearing slabs from adjacent empty tiles.

function _empty( x, y, force ) {
    if ( x < 0 || x >= that.width || y < 0 || y >= that.height ) return;

    var pos = that.pos(x, y);
    var d = that.grid[pos];

    if (d && (d & SLAB_MASK) && (force || !(d & APPLE_MASK))) {

        that.grid[pos] &= ~SLAB_MASK; // clear out slab

        // Clear next neighbor if this is empty tile
        if (that.grid[pos] == 0) {
            _empty(x, y - 1) // north
            _empty(x, y + 1) // south
            _empty(x - 1, y) // west
            _empty(x - 1, y - 1) // north west
            _empty(x - 1, y + 1) // south east
            _empty(x + 1, y) // east
            _empty(x + 1, y - 1) // north east
            _empty(x + 1, y + 1) // south east
        }
    }
}

Code is available at GitHub.

Car Game on HTML5

This is continuation to my previous blog post, that discussed how to make simple Slots machine with HTML5. The basic principle in this driving game demo is similar, but in addition to that it adds dynamic graphics using canvas.

HTML5 Drive

Try it out here: http://ikonen.me/examples/drive/

Game road is div with background image that is translated down and back up to create illusion of moving road. This same method is used in Slots machine.

Car spritecar2

Cars are drawn on canvas on top of the road so while road moves cars appear moving forward. Each click starts moving the player car from current point to the click point using smooth  Bezier curve. Car acceleration is increased or decreased depending on click location. Player car also has smoke plume animation which intensity depends on acceleration.

function _bezier_quad(t, p0, p1, p2) {
     return {
         x: (1 - t)*( (1 -t) * p0.x + t*p1.x) + t * ( (1 - t) * p1.x + t * p2.x),
         y: (1 - t)*( (1 -t) * p0.y + t*p1.y) + t * ( (1 - t) * p1.y + t * p2.y)
     }
 }

After car has travelled long enough, the update loop starts slowing down road speed and moves in sync  div that holds Finish text and flags.

finish

Collision detection is done on each update. In case player hits on other cars, the other cars may blurb random bubbles that are added as normal div elements with rounded borders. Element location is updated based on car location.

bubble

In collision both cars are bumped so player can make room by hitting to other cars. Game loop is run by animation frames, so it stops if browser window goes in background. This is not optimal but makes implementation simpler. Better option is to run update with setInterval and draw on animation frames.

...
    var that = this;
    (function gameLoop() {
        that.clear(); // clear canvas objects
        that.update(); // update game state
        that.draw(); // draw game
        requestAnimFrame( gameLoop );
    })();
...

Update and draw in separate loops. This is better solution for complex games because browser will always call setInterval callback but requestAnimFrame callback only if browser window is visible and active. Callback is not executed if browser window is minimized or it’s tab is on background. If you rely only on requestAnimFrame the game is paused on background.

    var that = this;
    setInterval(function() {
        that.update(); // update game state
    }, 1000/30); // 30 frames

    (function gameLoop() {
        that.clear(); // clear canvas objects
        that.draw(); // draw game
        requestAnimFrame( gameLoop );
    })();

Game code is available in GitHub.

Simple Slot machine game using HTML5 Part 1: Basics

UPDATE: See also Simple Slot machine game using HTML5 Part 2: Audio.

Here is overview on how to make simple Slot machine with HTML5. This demonstrates the basic structure of HTML5 game and how to use dynamically created graphics.

Slot machine has typically reels with images and player just initiates the action and waits until reels stop. 1 or more in single line usually determine the winning condition. In this game player wins if he or she gets more than one gold bar in row.

Here is view of the game.

Slots machine

You can try it out here http://ikonen.me/examples/slot/.

How it works

Slot machine is a single HTML page that includes the game code, webfont and jQuery. When loaded it runs SlotGame() function that initializes and runs the game.

HTML page has 3 narrow and tall HTML5 canvases, these are the reels. They are located inside div container “reels” that shows only a limited window at any time, hiding the rest of the canvases.

<div id="reels">
   <canvas id="canvas1" width="70" height="300"></canvas>;
   <canvas id="canvas2" width="70" height="300"></canvas>;
   <canvas id="canvas3" width="70" height="300"></canvas>;
</div>

On initialization, game preloads the 6 image assets. Preloading is simply done by creating Image object for each asset and listening its load event. Preloading is required, because otherwise game could not draw the reel canvases on initialization.

img = new Image()
img.src = "img/someimage.png"
img.addEventListener("load", function() {
   // image loaded
});

Game draws the pictures in random order on each canvas reel with shadow and slot separator bars.

ctx.save();
ctx.shadowColor = "rgba(0,0,0,0.5)";
ctx.shadowOffsetX = 5;
ctx.shadowOffsetY = 5;
ctx.shadowBlur = 5;
ctx.drawImage(asset.img, 3, i * SLOT_HEIGHT + IMAGE_TOP_MARGIN);
ctx.drawImage(asset.img, 3, (i + ITEM_COUNT) * SLOT_HEIGHT + IMAGE_TOP_MARGIN);
ctx.restore();
ctx.fillRect(0, i * SLOT_HEIGHT, 70, SLOT_SEPARATOR_HEIGHT);
ctx.fillRect(0, (i + ITEM_COUNT)  * SLOT_HEIGHT, 70, SLOT_SEPARATOR_HEIGHT);

The reels are not redrawn after this, but when moving they are simply translated with CSS3 transform downwards and when they reach threshold they are moved back to beginning. Threshold and reset offset is selected so that after the reset images are shown on same locations. This creates illusion of constantly rotating wheel. This is why images on borders are twice in the reel, so we avoid showing canvas bottom or top in any situation.
Click ‘Toggle Reels’ button on top left corner of the game page to toggle the reel visibility while it’s spinning, seeing reels in action makes explanation above much easier to understand

Here is image where reel container overflow is set to visible.

Reels

Game loop is simple, it starts when player clicks ‘Play’ and runs on every animation frame updating the reel locations based on game state and “draws” them on screen, or actually as explained earlier just translates their locations. Result is predetermined on each roll start and when each reel stop, its locked on the correct image. Update loop tries to make this when correct image is close to this location, so the jump is not too abrupt (see function _check_slot in slot.js for details).

Each browser has still different name for the transform so initialization code determines the correct CSS name and if browser has hardware accelerated 3d version.

this.vendor = 
  (/webkit/i).test(navigator.appVersion) ? '-webkit' :
  (/firefox/i).test(navigator.userAgent) ? '-moz' :
  (/msie/i).test(navigator.userAgent) ? 'ms' :
   'opera' in window ? '-o' : '';

this.cssTransform = this.vendor + '-transform';
this.has3d = ('WebKitCSSMatrix' in window &amp;&amp; 'm11' in new WebKitCSSMatrix())  
this.trnOpen       = 'translate' + (this.has3d ? '3d(' : '(');
this.trnClose      = this.has3d ? ',0)' : ')';
...
$('#someelement').css(this.cssTransform, this.trnOpen + '0px, ' + '-123px' + this.trnClose);</pre>

Complete code is available on Github.

Continue to Simple Slot machine game using HTML5 Part 2: Audio

Cross Domain data channel with HTML5 Canvas

Standard Ajax is restricted to single origin policy so JSONP is the de-facto way for exchanging data with cross-domain sources and it works pretty well. Alternative, though bit hacky way, is to use HTML5 Canvas as cross domain work-around using pseudo Images as “covert channel”.

Basic idea is simple, javascript in client requests image file from 3rd party site where server encodes a data to the Image, client can use cookies and url parameters to identify itself as desired. Then client renders image, and decodes the data from the image pixels.

Backend

Backend needs to be able to construct images with custom pixel level data, in this example we use Node.js and canvas module that is server side HTML5 Canvas implementation based on Cairo graphics library.

This function accepts any object and returns canvas object that contains the objects JSON presentation encoded in image pixels.

function encodeDataToImage( data ) {

	// Convert data to binary buffer while being utf-8
	var s = encodeURIComponent( JSON.stringify(data) );
	var buffer = new Buffer(s, 'utf8');
	var pixelc = (buffer.length / 3) + (buffer.length % 3 ? 1 : 0)

	// Encode data as PNG image
	var Canvas = require('canvas');
	var canvas = new Canvas(pixelc, 1)
	var ctx = canvas.getContext('2d');
	var imgdata = ctx.getImageData(0, 0, pixelc, 1);

	for (var i=0, k=0; i < pixelc * 4; i += 4 ) {
		imgdata.data[i + 3] = 0xFF; // set alpha to full opaque
		for (var j=0; j < 3 && k < buffer.length; k++, j++ ) {
			imgdata.data[i + j] = buffer[k];
		}
	}
	// set "image" data
	ctx.putImageData(imgdata, 0, 0);
	return canvas;
}

Define xd request handler that builds and sends the data coded image to the client. (Example in Express.js).

someapp.get('/xd', function(req, res ) {
    // do here something with query or cookies, like resolve uid and set
    // data.
    // Example data
    var data = { a: 1, en: 'owl', fi: 'pöllö', es: 'búho', uid: req.query.uid }

    var canvas = encodeDataToImage( data );
    var img = canvas.toBuffer();
    res.contentType('png');
    res.header('Content-Length', img.length);
    res.send( img );
});

Browser
At browser side load the image and decode it back to object

function queryXD( query, callback ) {

	var img = new Image();
	img.src = 'http://some.site.example.com/xd?' + query;
	img.addEventListener('load', function() {

		// Image loaded, create temporary canvas
		var canvas = document.createElement('canvas');
		var ctx = canvas.getContext('2d');

		// draw image on canvas
		canvas.width = img.width;
		canvas.height = img.height;
		ctx.drawImage( img, 0, 0 );

		// collect bytes from image pixels
		var bytes = [];
		var imgdata = ctx.getImageData(0, 0, img.width, img.height);
		for (var i=0; i < img.width * 4; i++ ) {
			if ( i && (i + 1) % 4 == 0) {
				i++;
			}
			var b = imgdata.data[i];
			if (!b) {
				break;
			}
			bytes.push( b );
		}

		// convert bytes to string and parse JSON
		var s = decodeURIComponent( String.fromCharCode.apply(null, bytes) );
		var data = JSON.parse(s);

		callback(false, data);
	}, false);

        // image failed to load
	img.addEventListener('error', function(err) {
		callback(err);
	}, false);
}

And now its simple to do cross domain data exchange like

queryXD('uid=2134', function(err, data) {
   alert(data.en + ' is ' + data.fi + ' in Finnish and ' + data.es + ' in Spanish');
});

Caveats
Proxies and browsers like to cache the images, so use every time unique dummy parameter to force fetch.

HTML5 Canvas Layout and Mobile Devices

Common problem with Canvas and mobile devices is how to get the canvas to fill the browser window properly. This can be tricky and require lots of tweaking and testing with different devices to get it exactly right.

Even if you get the size correctly defined, the rotation is another hurdle and the layout could break after orientation change or two.

I wrote an example of simple layout page, that should work both on desktops and mobile devices Android (>2.2) and iPhone/iPad. It should appear as following layout in all browsers shown here in the iPhone screenshots and not break on resize or orientation change.

portrait

 

The layout works also after rotation.

layout_l

Page defines a canvas (green), that occupies most of the screen and under that a fixed height div (yellow) containing ‘Some Text Here’. On every resize the code draws black rectangle that is -10 pixels short from each canvas border and writes number of orientation changes and the resize events for debugging purposes. The document background is defined blue to reveal possible unwanted overflows.

How it works?

DOM/CSS

First the meta elements tell to mobile devices how to handle the page. No scaling and width is fixed to device width.

<meta name="viewport" content="user-scalable=no, initial-scale=1.0, maximum-scale=1.0, width=device-width">
<meta name="apple-mobile-web-app-capable" content="yes">

The document is wrapped in single div (“container”) that contains the canvas and the fixed height div.

<body>
   <div id="container">
     <canvas id="canvas">HTML5 Canvas not supported.</canvas>
     <div id="fix">Some Text Here</div>
   </div>
...

Container is forced to fill the browser window by CSS rule that defines overflow as auto and width/height 100%.

body,html
{
    height: 100%;
    margin: 0;
    padding: 0;
    color: black;
}
#container
{
    width: 100%;
    height: 100%;
    overflow: auto;
}

Canvas element is inside the container and has no initial height and width. It is defined as display block in CSS to avoid unwanted padding or margins. Canvas default display is inline, that is something you almost never want.

#container canvas {
    vertical-align: top;
    display: block;
    overflow: auto;
}

Finally div (“fix”) is defined with fixed height

#fix {
    background: yellow;
    height: 20px;
}

This is not enough though, and some JS handling is required for resize and the orientation change.

Javascript

The JS listens both timeout and orientation change events and installs a timeout function that gets cancelled if browser sends several events rapidly.

var resizeTimeout;
$(window).resize(function() {
    clearTimeout(resizeTimeout);
    resizeTimeout = setTimeout(resizeCanvas, 100);
});

var otimeout;
window.onorientationchange = function() {
    clearTimeout(otimeout);
    otimeout = setTimeout(orientationChange, 50);
}

Orientation change listener does nothing important, it just updates the counter for debugging purposes.

The resizeCanvas is more involved. When browser is iPhone it first increases the container height 60 pixels higher than the browser window height. This makes possible to scroll the window down and hide the iPhone Safari address bar.

if (ios) {
    // increase height to get rid off ios address bar
    $("#container").height($(window).height() + 60)
    setTimeout(function() { window.scrollTo(0, 1);  }, 100);
}

Then it gets the container width and height, that are height and width of the browser window.

var width = $("#container").width();
var height = $("#container").height();

And finally forces the canvas size and width to the required. The height is subtracted by 20 to leave room for the fixed height div.

cheight = height - 20; // subtract the fix div height
cwidth = width;

// set canvas width and height
$("#canvas").attr('width', cwidth);
$("#canvas").attr('height', cheight);

There could be better way to do this, but at least this seems to be pretty robust and works in all major desktop and mobile browsers.

Code is available in Github.