Unity C++ Native Plugin Examples

Unity scripting environment runs C# Mono container which supports native C/C++ plugins with Mono PInvoke. This allows easy integration of native library functions to both pass and receive data between Unity managed C# and native platform code.

Basic concept is simple. Native function descriptor that specify calling convention and dynamically loaded library (DDL) tell Mono how it’s parameters and returns values should be converted. Mono handles things mostly automatically from there on.

Some care is required to do this efficiently. C# PInvoke defines marshaling as a protocol to serialize and deserialize managed objects back and forth towards native code. This may generate lot of overhead. Mono garbage collector might accidentally destroy memory of an object while it’s used in the native code space.

Example C++ code is for a native Win32 plugin but is trivial to port to any platform.

Basic Data

Passing simple integral types, integers and arrays of them is straightforward.

Native c code.

extern "C" {

#define PLUGINEX(rtype) UNITY_INTERFACE_EXPORT rtype UNITY_INTERFACE_API

    PLUGINEX(int) ReturnInt()
    {
        return 0xBABE;
    }

    PLUGINEX(void) AcceptArray1(char *arr, int length)
    {
        for (int i = 0; i < length; i++) {
           arr[i] = 'A' + i;
        }
    }
}

C# declarations and example calling code.

[DllImport("ptest")]
private static extern int ReturnInt();

[DllImport("ptest")]
private static extern void AcceptArray1([In, Out] byte[] arr, int length);

void TestIntegral() {
    // return int
    print(ReturnInt());

    // accept byte array, uses marshaling to pass array back and forth
    byte[] arr1 = { 0, 0, 0 };
    AcceptArray1(arr1, arr1.Length);
    for (int i = 0; i < arr1.Length; i++)
    {
        print("arr" + i + "=" + arr1[i]);
    }
}
Strings

Strings are passed and returned as character arrays. It’s possible to use automatic marshaling conversions and return dynamically allocated strings that will be automatically deallocated.

Native c code.

extern "C" {

#define PLUGINEX(rtype) UNITY_INTERFACE_EXPORT rtype UNITY_INTERFACE_API

	PLUGINEX(bool) AcceptStr(LPCSTR pStr)
	{
		return !strcmp(pStr, "FOO");
	}
	PLUGINEX(LPSTR) ReturnDynamicStr()
	{		
		LPSTR str = (LPSTR)CoTaskMemAlloc(512);
		strcpy_s(str, 512, "Dynamic string");
		return str;
	}

	PLUGINEX(LPCSTR) ReturnConstStr()
	{		
		return "Constant string";
	}
}

C# declarations and example calling code.

[DllImport("ptest")]
private static extern bool AcceptStr([MarshalAs(UnmanagedType.LPStr)] string ansiStr);

// automatically deallocates the return string with CoTaskMemFree
[DllImport("ptest")]
[return: MarshalAs(UnmanagedType.LPStr)]
private static extern string ReturnDynamicStr();

[DllImport("ptest")]
private static extern IntPtr ReturnConstStr();


void TestStrings() {
    // accept string
    bool r1 = AcceptStr("BAR");
    bool r2 = AcceptStr("FOO");
    print("r1=" + r1); // r1=false
    print("r2=" + r2); // r1=true

    // return dynamically allocated string
    string s1 = ReturnDynamicStr();
    print("s1=" + s1);

    // return constant string
    string s2 = Marshal.PtrToStringAnsi(ReturnConstStr());
    print("s2=" + s2);
}
Arrays

Both dynamic and constant arrays can be supported but some helper functions are needed.

Native c code.

extern "C" {

#define PLUGINEX(rtype) UNITY_INTERFACE_EXPORT rtype UNITY_INTERFACE_API

	PLUGINEX(void) AcceptArray1(char *arr, int length)
	{
		for (int i = 0; i < length; i++) {
			arr[i] = 'A' + i;
		}
	}

	PLUGINEX(void) AcceptArray2(char *arr, int length)
	{
		for (int i = 0; i < length; i++) {
			arr[i] = 'A' + i;
		}
	}

	PLUGINEX(int) AcceptStrArray(const char* const *strArray, int size)
	{
		int total = 0;
		for (int i = 0; i < size; i++) {
			auto str = strArray[i];
			total += (int)strlen(str);
		}
		// return total length of the strings in the array to demonstrate that
		// it was passed correctly
		return total;
	}

	PLUGINEX(LPBYTE) ReturnDynamicByteArray(int &pSize)
	{
		pSize = 0xFF;
		LPBYTE pData = (LPBYTE)CoTaskMemAlloc(pSize);

		// fill with example data
		for (int i = 0; i < pSize; i++) {
			pData[i] = i + 1;
		}

		return pData;
	}

	PLUGINEX(LPSTR*) ReturnDynamicStrArray(int &pSize)
	{
		// Allocate an array with pointers to 3 dynamically allocated strings
		pSize = 3;
		LPSTR* pData = (LPSTR*)CoTaskMemAlloc((pSize)*sizeof(LPSTR));		
		pData[0] = (LPSTR)CoTaskMemAlloc(128);
		pData[1] = (LPSTR)CoTaskMemAlloc(128);
		pData[2] = (LPSTR)CoTaskMemAlloc(128);

		strcpy_s(pData[0], 128, "String 1");
		strcpy_s(pData[1], 128, "String 2");
		strcpy_s(pData[2], 128, "String 3");

		return pData;
	}
}

C# declarations and example calling code.


[DllImport("ptest")]
private static extern void AcceptArray1(IntPtr arr, int length);

[DllImport("ptest")]
private static extern void AcceptArray2(IntPtr arr, int length);

[DllImport("ptest")]
private static extern int AcceptStrArray(IntPtr array, int size);

[DllImport("ptest")]
private static extern IntPtr ReturnDynamicByteArray(ref int size);

[DllImport("ptest")]
private static extern IntPtr ReturnDynamicStrArray(ref int size);

///// Helper functions for marshalling /////

// Convert and copy array of strings to raw memory
private static IntPtr MarshalStringArray(string[] strArr)
{
    IntPtr[] dataArr = new IntPtr[strArr.Length];
    for (int i = 0; i < strArr.Length; i++)
    {
        dataArr[i] = Marshal.StringToCoTaskMemAnsi(strArr[i]);
    }
    IntPtr dataNative = Marshal.AllocCoTaskMem(Marshal.SizeOf(typeof(IntPtr)) * strArr.Length);
    Marshal.Copy(dataArr, 0, dataNative, dataArr.Length);

    return dataNative;
}

// Decodes string array from raw pointer
private static string[] MarshalStringArray(IntPtr dataPtr, int arraySize)
{
    var dataPtrArray = new IntPtr[arraySize];
    var strArray = new String[arraySize];
    Marshal.Copy(dataPtr, dataPtrArray, 0, arraySize);
    for (int i = 0; i < arraySize; i++)
    {
        strArray[i] = Marshal.PtrToStringAnsi(dataPtrArray[i]);
        Marshal.FreeCoTaskMem(dataPtrArray[i]);
    }
    Marshal.FreeCoTaskMem(dataPtr);
    return strArray;
}

// Dellocates encoded string array
private static void CleanUpNativeStrArray(IntPtr dataPtr, int arraySize)
{
    var dataPtrArray = new IntPtr[arraySize];
    Marshal.Copy(dataPtr, dataPtrArray, 0, arraySize);
    for (int i = 0; i < arraySize; i++)
    {
        Marshal.FreeCoTaskMem(dataPtrArray[i]);
    }
    Marshal.FreeCoTaskMem(dataPtr);
}

void TestArrays() {

    // accept byte array, uses marshalling to pass array back and forth
    byte[] arr1 = { 0, 0, 0 };
    AcceptArray1(arr1, arr1.Length);
    for (int i = 0; i < arr1.Length; i++)
    {
        print("arr" + i + "=" + arr1[i]);
    }

    // accept byte array, passes no-copy raw memory pointer
    byte[] arr2 = { 0, 0, 0 };
    GCHandle h = GCHandle.Alloc(arr2, GCHandleType.Pinned);
    AcceptArray2(h.AddrOfPinnedObject(), arr2.Length);
    for (int i = 0; i < arr2.Length; i++)
    {
        print("arr" + i + "=" + arr2[i]);
    }
    h.Free();

    // return dynamically allocated byte array
    int arraySize = 0;
    IntPtr dataPtr = ReturnDynamicByteArray(ref arraySize);
    byte[] data = new byte[arraySize];
    Marshal.Copy(dataPtr, data, 0, arraySize);
    Marshal.FreeCoTaskMem(dataPtr); // deallocate unmanaged memory
    print("data["+arraySize+"] = [" + data[0] + ", " + data[1] + ", " + data[2] + ",...]");

    // return dynamically allocated string array
    arraySize = 0;
    dataPtr = ReturnDynamicStrArray(ref arraySize);
    String[] strArray = MarshalStringArray(dataPtr, arraySize);
    print("strArray["+arraySize+"] = [" + String.Join(",", strArray) + "]");

    // string array as parameter
    dataPtr = MarshalStringArray(new String[] { "foo1", "foo2", "foo3" });
    int len = AcceptStrArray(dataPtr, arraySize);
    print("len=" + len);
    CleanUpNativeStrArray(dataPtr, arraySize);

}
Structures and Arrays of Structures

Array handling is similar to the string arrays. Most objects can be passed as is but if they have array properties those must be of fixed size.

Native c code.

extern "C" {

#define PLUGINEX(rtype) UNITY_INTERFACE_EXPORT rtype UNITY_INTERFACE_API

	struct ExampleStruct {
		INT16 val1;
		INT32 array1[3];
		INT16 array2len;
		INT32 array2[10];
		LPSTR str1;
	};

	PLUGINEX(int) AcceptStruct(ExampleStruct &s)
	{
		// Modify struct
		s.val1 -= 1111;
		for (int i= 0; i < 3; i++) {
			s.array1[i] += 1;			
		}
		for (int i = 0; i < s.array2len; i++) {
			s.array2[i] += 10;
		}
		// return length of the string in the argument struct to demonstrate that
		// it was passed correctly
		return (int)strlen(s.str1);
	}

	struct ExamplePoint {
		FLOAT x;
		FLOAT y;
		FLOAT z;
	};

	PLUGINEX(ExamplePoint *) ReturnArrayOfPoints(int &size)
	{
		size = 4;
		ExamplePoint *pointArr = (ExamplePoint*)CoTaskMemAlloc(sizeof(ExamplePoint) * size);

		// fill with some example data
		for (int i = 0; i < size; i++) {
			pointArr[i] = { i + 0.1f, i + 0.2f, i + 0.3f };
		}
		return pointArr;
	}

	// this return type is blittable
	// https://stackoverflow.com/questions/10320502/c-sharp-calling-c-function-that-returns-struct-with-fixed-size-char-array
	//
	PLUGINEX(ExamplePoint) ReturnStruct()
	{		
		return { 1, 2, 3 };
	}	
}

C# declarations and example calling code.

[StructLayout(LayoutKind.Sequential)]
public struct ExamplePoint
{
    public float x;
    public float y;
    public float z;

    // for debugging
    public override String ToString()
    {
        return "{" + x + ","+ y + "," + z + "}";
    }
}

[DllImport("ptest")]
private static extern IntPtr ReturnArrayOfPoints(ref int size);

[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct ExampleStruct
{
    public UInt16 val1;
    [MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 3)]
    public UInt32[] array1;
    public UInt16 array2len;
    [MarshalAsAttribute(UnmanagedType.ByValArray, SizeConst = 10)]
    public UInt32[] array2;
    [MarshalAs(UnmanagedType.LPStr)]
    public string str1;
}

[DllImport("ptest")]
private static extern int AcceptStruct(ref ExampleStruct s);

[DllImport("ptest")]
private static extern ExamplePoint ReturnStruct(); 


void TestStructures() {
    // Structure as parameter
    ExampleStruct s = new ExampleStruct
    {
        val1 = 9999,
        array1 = new UInt32[3],
        array2 = new UInt32[10]
    };
    s.array1[0] = 1;
    s.array1[1] = 2;
    s.array1[2] = 3;
    s.array2len = 5;
    s.array2[0] = 10;
    s.array2[1] = 11;
    s.array2[2] = 12;
    s.array2[3] = 13;
    s.array2[4] = 14;
    s.str1 = "Cat is a feline";

    len = AcceptStruct(ref s);
    print("s.val1=" + s.val1 + " len=" + len);

    // return struct
    ExamplePoint p = ReturnStruct();

    // Marshal array of point objects
    arraySize = 0;
    dataPtr = ReturnArrayOfPoints(ref arraySize);
    ExamplePoint[] pointArr = new ExamplePoint[arraySize];

    // memory layout
    // |float|float|float|float|float|float|float|float|float|float..
    // |   ExamplePoint0 |   ExamplePoint1 |   ExamplePoint2 |
    int offset = 0;
    int pointSize = Marshal.SizeOf(typeof(ExamplePoint));
    for(int i=0; i < arraySize; i++)
    {
        pointArr[i] = (ExamplePoint)Marshal.PtrToStructure(new IntPtr(dataPtr.ToInt32() + offset), typeof(ExamplePoint));
        offset += pointSize;
    }
    print("pointArr["+arraySize+"]=["+pointArr[0]+", "+pointArr[1]+",...]");
    Marshal.FreeCoTaskMem(dataPtr);

}

Many of these examples can be done in cleaner way using MarshalAsAttribute. Code above does show what is happening under the hood.

Get full implementation ftom https://github.com/tikonen/blog/tree/master/unityplugin

Quickstart for SQLite in Unity

I had some troubles setting up SQLite for my Unity project for Android and iOS app. Let’s hope this quickstart helps you faster up to speed with your mobile app.

DISCLAIMER. This example is for OS/X development environment and Android and iOS builds. Never tried this for Windows but I guess installing sqlite3.dll should do the trick.

Step 1. Get wrapper API

First, get the SQLite library by @busta117 from Github: https://github.com/Busta117/SQLiteUnityKit

Step 2. Copy API files on your project

Copy the DataTable.cs and SqliteDatabase.cs somewhere under your projects Assets/Scripts/ folder. If you build also for android, then copy libsqlite3.so in your projects Assets/Plugins/Android/ folder. iOS does not need plugin as it has native support for sqlite.

Step 3. Create default database

This is the database that should contain the tables you need with any default data you may want to have. Default database is used to bootstrap the actual in app database.

Create folder Assets/StreamingAssets/. Then create your default template database with sqlite3.

$ sqlite3 Assets/StreamingAssets/default.db
SQLite version 3.8.8.3 2015-02-25 13:29:11
Enter ".help" for usage hints.
sqlite> create table example (
   ...> name string,
   ...> dummy int
   ...> );
sqlite> .schema example
CREATE TABLE example (
name string,
dummy int
);
sqlite> insert into example values ("hello world", 1);
sqlite> select * from example;
hello world|1
sqlite>.quit
$

Now you have default database and your project should have files like these.

./Assets/Plugins/Android/libsqlite3.so
./Assets/Scripts/SQLiteUnityKit/DataTable.cs
./Assets/Scripts/SQLiteUnityKit/SqliteDatabase.cs
./Assets/StreamingAssets/default.db

Step 4. Database initialization code.

Initialize the database in your main scripts Awake() method. This checks if database already exists and if not, it copies the default db as template.


SqliteDatabase sqlDB;

void Awake() 
{
    string dbPath = System.IO.Path.Combine (Application.persistentDataPath, "game.db");
    var dbTemplatePath = System.IO.Path.Combine(Application.streamingAssetsPath, "default.db");

    if (!System.IO.File.Exists(dbPath)) {
        // game database does not exists, copy default db as template
        if (Application.platform == RuntimePlatform.Android)
        {
            // Must use WWW for streaming asset
            WWW reader = new WWW(dbTemplatePath);
            while ( !reader.isDone) {}
            System.IO.File.WriteAllBytes(dbPath, reader.bytes);
        } else {
            System.IO.File.Copy(dbTemplatePath, dbPath, true);
        }		
    }
    sqlDB = new SqliteDatabase(dbPath);
}

You can use the script execution order setting to ensure that this code is always executed first.

Step 5. Use the database!

API supports normal selects, inserts and updates.

var result = sqlDB.ExecuteQuery("SELECT * FROM example");
var row = result.Rows[0];
print("name=" + (string)row["name"]);
print("dummy=" + (int)row["dummy"]);

API is simple to use, check detailed documentation from the https://github.com/Busta117/SQLiteUnityKit.

Finding maximal rectangles in a grid

Maximal rectangles problem is to find rectangles inside an area that have maximum possible surface area. This is often needed in games to show the player where in area a certain item can fit etc.. There are many ways to approach the problem, and for some cases (such as areas defined by polygon) they can get complex to implement. Here is one relatively simple method for looking up those maximum rectangles inside an area in a common grid. It’s pretty efficient and does support following restrictions

  • Rectangles can have minimum size (e.g. larger than 2×2).
  • There can be more than one maximum rectangle in the area. (in case two large areas are joined with narrow path)

First part is defining function that looks up rectangle dimensions. This function starts from a tile that will be left top corner of rectangle and returns its width and height. The listings here are in pseudocode.

Listing 1. Get rectangle size

FUNC get_rectangle_size( tile, tiles )
BEGIN
    # find rectangle width
    width = 1
    WHILE contains( (tile.x + width, c.y,  tiles )
      width = width + 1
   
    # find rectangle height
    height = 0
    stop = false
    REPEAT
       height = height + 1
       LOOP FROM x = tile.x TO tile.x + width
         IF NOT contains( (x, tile.y + height), tiles )
         THEN
            stop = true
            BREAK
         ELSE
            x = x + 1
    UNTIL stop  

    RETURN (width, height)
END

Function finds the maximum width of the rectangle starting from the left corner and working right (lines 5-6), then it finds rectangle height by looking up rows of same width (lines 11-20).

The actual algorithm that finds the rectangles accepts list of grid tiles as input and outputs the found rectangles that are maximally sized. The pseudocode listing here assumes some helper functions for simplicity.

  • sort ( list, compare ) – Sorts list in place by compare function
  • add( item, list ) – Adds item to a list
  • remove (item, list) – Removes item from list
  • contains( item, list ) – Returns true if item is found from list
  • length( list ) – Returns lists length
  • overlap (rect1, rect2) – Returns true if two rectangles overlap

Listing 2. Find maximal rectangles. Unoptimized for readability!

FUNC find_tiles( tiles )  # the list of tiles as (x,y) tuples
BEGIN
  rectangles = []  # list of found rectangles

  # sort with compare order by y and x.
  sort(tiles,  (a, b) => a.y == b.y ? a.x - b.x : a.y - b.y)

  WHILE length(tiles) > 0
    c = head(tiles)   # take first item from the sorted list
    
    # get rectangle size
    (width, height) = get_rectangle_size( c, tiles ) 

    # remove tiles that can not be part of any larger rectangle
    LOOP FROM x = c.x TO c.x + width
      LOOP FROM y = c.y TO c.y + height
         IF NOT contains( (c.x - 1, y, tiles) AND
            NOT contains( (c.x + width, y), tiles) AND
            NOT contains( (x, c.y - 1), tiles) AND
            NOT contains( (x, c.y + height), tiles)
         THEN
            remove( (x, y), tiles )
 
    # if big enough rectangle add it to list
    IF width > 1 AND height > 1
    THEN
      add ( (c.x, c.y, width, height), rectangles ) 

  # sort rectangles by area size
  sort( rectangles, (r1, r2) => r2.width * r2.height - r1.width * r1.height )

  # remove overlapping rectangles
  i = 0
  WHILE i < length(rectangles) - 1
    LOOP FROM j = length(rectangles) - 1 TO i + 1 # descending order 
        IF overlaps(rectangles[i], rectangles[j])
        THEN
          remove(rectangles[j], rectangles)
    i = i + 1

  RETURN rectangles
END

Here is example how the algorithm works, here white area presents the working set of tiles that algorithm works on, white presents the tiles that are in list tiles.
step1-rect

Algorithm sorts first the tiles in top-to-bottom and left-to-right order so it starts from top left tile. The pink denotes the current working tile c. The green represents the tiles that belong to rectangle as determined by call to get_rectangle_size.
step2-rect

Next step is to remove tiles from working set that can not be part of any other rectangle. This is determined by checking if the row and column of tile are bounded inside the current rectangle. The purple presents tiles that were removed from the working set tiles. Found rectangle is added in the list rectangles if it’s large enough (in this case larger than 2×2).
Then loop is executed again with next corner tile c.
step3-rect

Next iteration of the main loop returns another rectangle and closed tiles are removed and rectangle added to list.
step4-rect

Final loop of the rectangle. There are no more tiles to work so main loop stops.
step5-rect

Finally algorithm sorts the found rectangles in descending surface area order and removes overlap. The resulting rectangles are returned as output.

Class Persistence in Unity

Games need to store some persistent data like high scores and progress between the game sessions. Fortunately Unity gives us PlayerPrefs class that is essentially a persistent hash map.

Reading and writing values with PlayerPrefs is as simple as calling get and set.

// saving data
PlayerPrefs.SetInt("foobar", 10);
PlayerPrefs.SetString("something", "foo");
PlayerPrefs.Save();
...
// reading data
if(PlayerPrefs.HasKey("foobar")) {
    int foo = PlayerPrefs.GetInt("foobar");
}

It has its limitations, only strings and numbers can be stored and that makes more complex data lot more difficult to maintain.

What we can do is to write simple utility that can be used to serialize classes to strings that can then be read and written with PlayerPrefs. SerializerUtil is static class with two methods to Load and Write object. In case loading fails it returns default value of the data, usually null.

using System;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;

public static class SerializerUtil {
	static BinaryFormatter bf = new BinaryFormatter ();
	
	public static T LoadObject<T>(string key)
	{
		if (!PlayerPrefs.HasKey(key))
			return default(T);
		
		try {	
			string tmp = PlayerPrefs.GetString(key);
			MemoryStream dataStream = new MemoryStream(Convert.FromBase64String(tmp));
			return (T)bf.Deserialize(dataStream);
			
		} catch (Exception e) {
			Debug.Log("Failed to read "+ key+ " err:" + e.Message);
			return default(T);
		}				
	}
	
	public static void SaveObject<T>(string key, T dataObject)
	{
		MemoryStream memoryStream = new MemoryStream ();
		bf.Serialize (memoryStream, dataObject);
		string tmp = Convert.ToBase64String (memoryStream.ToArray ());
		PlayerPrefs.SetString ( key, tmp);
	}
}

To load and save your classes, declare them as Serializable.

[Serializable]
public class PlayerData {
	public int points;
	public string name; 		
}

You can then easily store instances of the class.

var data = new PlayerData();
data.points = 50;
data.name = "Teemu";

SerializerUtil.SaveObject("player1", data);

Reading classes is also simple

PlayerData data;
data = SerializerUtil.LoadObject<PlayerData>("player1");
// data.points is 50
// data.name is "Teemu"

Classes can be also more complicated, you can add member functions and also exclude some member variables from serialization. It’s also possible to define member function that will be called after serialization, it’s good way to init class after deserialization from disk.

[Serializable]
public class PlayerData : IDeserializationCallback {
	public int points;
	public string name;
    
	// serialization ignores this member variable
	[NonSerialized] Dictionary<int, int> progress = new Dictionary<int, int>();

	// constructor is called only on new instances
	public PlayerData() {
		points = 0;
		name = "Anon";
	}
	
	// serializable class can have member methods as usual
	public bool ValidName() 
	{
		return name.Trim().Length > 4; 
	}

	// Called only on deserialized classes
	void IDeserializationCallback.OnDeserialization(System.Object sender) 
	{
	    // do your init stuff here
	    progress = new Dictionary<int, int>();	
	}
}

Caveats

This serialization does not support versioning. You can not read the stored instance anymore if you change the class members as the LoadObject will fail to deserialize the data.

You need to add following environment variable to force Mono runtime to use reflection instead JIT. Otherwise the serialization will fail on iOS devices. Do this before doing any loading or saving of classes.

void Awake() {
    Environment.SetEnvironmentVariable("MONO_REFLECTION_SERIALIZER", "yes");
    ...

Easy HTML5 canvas and CSS Sprite texture atlas

Texture atlas is a collection of images, all composed in single large image. The game or HTML5 app can load this single image instead of wasting time for requesting every small image separately.

I wrote tool that can be used to package collection of images to single atlas and let web app use that conveniently both as CSS sprite or in HTML5 canvas drawing.

Tool itself is Python script that uses ImageMagick commands to read and write images. The output is the atlas image, CSS, raw JSON and Javascript import file. CSS file defines CSS sprite for each image. JSON is raw data of each images position in the atlas, Javascript import file can be included on HTML page and it loads the atlas position info into global variable.

Setup

Get script from here: https://raw.github.com/tikonen/blog/master/packer/packer.py

Make it executable

$ chmod +x packer.py

Verify that all is ok

~/work/blog/packer $ ./packer.py -h
usage: packer.py [-h] [-o OUTFILE] [-jo JSONOUTFILE] [-jso JSOUTFILE]
                 [-co CSSOUTFILE] [-p PAD] [-mw WIDTH] [-mh HEIGHT]
                 FILE [FILE ...]

Packs images to atlas. Uses ImageMagick to parse and compose the images

positional arguments:
  FILE             Image file

optional arguments:
  -h, --help       show this help message and exit
  -o OUTFILE       Output atlas file
  -jo JSONOUTFILE  Output atlas json file
  -jso JSOUTFILE   Output atlas import js file
  -co CSSOUTFILE   Output atlas css file
  -p PAD           Padding
  -mw WIDTH        Maximum width
  -mh HEIGHT       Maximum height

Install ImageMagick easily on OS/X with Macports or Homebrew.

$ sudo port install ImageMagick

Verify that ImageMagick installation works

$ identify --version
Version: ImageMagick 6.8.7-3 2013-10-28 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2013 ImageMagick Studio LLC
Features: DPC
Delegates: bzlib djvu fftw fontconfig freetype gslib jng jpeg lcms ltdl lzma png ps png tiff webp x xml zlib

Usage

You use script by giving image file as arguments and defining the desired output files and maximum image dimensions

For example:

$ ./packer.py img/*.png -o sprites.png

This would produce sprites.png, sprites.json, sprites.css and sprites.json.js

Options
Following options are supported

  • -p PAD
    Defines padding. The amount of empty pixels around each image. This prevents scaling artifacts on HTML canvas use when drawing from decimal source coordinates.
  • -mw WIDTH,-mh HEIGHT
    Maximum width and height of output image.
  • -o FILE
    Output image file.
  • -jo FILE, -jso FILE, -co FILE
    Output JS import, JSON and CSS file.

Example

We have following small images that are needed in our web app. Loading them all independently would take 5 HTTP requests.

  • button_minus.png
  • button_plus.png
  • cannon_marker.png
  • pause.png
  • status_bar.png

montage

Running tool converts them to single atlas

~/work/blog/packer $ ./packer.py example/pics/* -o example/html/sprites.png -mw 512
Checking ImageMagick
Found: Version: ImageMagick 6.8.7-3 2013-10-28 Q16 http://www.imagemagick.org
===========================
Resolving file dimensions
button_minus.png -> 53x43
button_plus.png -> 53x42
cannon_marker.png -> 43x28
pause.png -> 53x42
status_bar.png -> 496x74
===========================
fitting 5 images, padding 1
successfully fitted 5 images to 496x118 padding 1
Wrote: atlas to example/html/sprites.png
Wrote json to example/html/sprites.json
Wrote js to example/html/sprites.json.js
Wrote css to example/html/sprites.css

In this example we defined maximum width of 512 pixels. The resulting image is cropped to minimum size 496×118.

sprites
This atlas can be used now in two different ways on the web page.

CSS Sprite

Web page includes the generated CSS file and defines class by filename (e.g. “bg-sprites status_bar” for each element that uses sprite.

<!DOCTYPE html>
<html>
<head>
    <title>Example</title>
    <link rel="stylesheet" type="text/css" href="sprites.css">
    <style type="text/css">
        .bg-sprites {
            color: white;
            text-align: center;
        }
    </style>
</head>
<body>
    <div>
        <h1>Example</h1>
        <div>
            <h2>Cannon - simple sprite with image</h2>
            <div class="bg-sprites cannon_marker">
            </div>
            <h2>Status bar - sprite with text inside</h2>
            <div class="bg-sprites status_bar">
                <span style="font-size:xx-large;position:relative;top:15px;">Here is some text</span>
            </div>
            <h2>Buttons - list</h2>
            <div>
                <span style="float:left;margin:5px;" class="bg-sprites button_minus"></span>
                <span style="float:left;margin:5px;" class="bg-sprites button_plus"></span>
                <span style="float:left;margin:5px;" class="bg-sprites pause"></span>
            </div>
        </div>
    </div>
</body>

This renders following page:
example css sprite

See example page here: http://ikonen.me/examples/packer/example_css.html

HTML5 Canvas

Another option is to use it with HTML canvas. Page links the generated JSON import script and loads the image.

<!DOCTYPE html>
<html>
<head>
    <title>Example</title>
    <style type="text/css">
        canvas {
            width: 600px;
            height: 400px;
        }
    </style>
</head>
<body>
    <div>
        <h1>Canvas Example</h1>
        <div>
            <canvas id="thecanvas" width="600" height="400"></canvas>
        </div>
    </div>
  <script src="http://ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js" ></script>
  <script type="text/javascript" src="sprites.json.js"></script>
  <script type="text/javascript">
      $(function() {
          var canvas = document.getElementById('thecanvas');
          var ctx = canvas.getContext('2d');
          ctx.strokeStyle = 'hotpink';
          ctx.strokeRect(0, 0, 600, 400);
          ctx.font = '20px Arial';

          // Load image and the json that defines locations
          var sprites = new Image();
          sprites.src = 'sprites.png';
          sprites.addEventListener("load", function() {
                  var assets = bg_sprites; // imported by sprites.json.js

                  // draw them all
                  var xoffset = 5,
                      yoffset = 25;

                  for (var pic in assets) {
                      var asset = assets[pic];
                      ctx.fillText(pic, xoffset, yoffset-3);

                      // draw image from sprite
                      ctx.drawImage(sprites, asset.x, asset.y, asset.w, asset.h, xoffset, yoffset, asset.w, asset.h);

                      yoffset += asset.h + 20;
                  }
          }, false);
      });
  </script>
</body>
</html>

This renders following page:

example canvas

See example page here: http://ikonen.me/examples/packer/example_canvas.html

Get complete example from Github.

Reduce mp3 file size for app and web use

Typical 2 minutes piece of music is roughly 2-4MB in common mp3 format. This is size for usual quality that is stereo audio 256 kbps and 44.1 kHz sample rate. Tough the file size is not that large, it will slow down loading HTML5 game or mobile app, especially on wireless networks.

Web HTML5 or mobile game applications do not need nearly as high quality and the size can be reduced to ~400-700kB without too much audible quality penalty. Difference can be noticed if you listen for it, but it’s unlikely that users do notice anything, as they have nothing to compare it against. This also depends lot of the music, sharp high frequency sounds tend to distort first.

MP3/WAV to reduced size MP3

Easy way to convert mp3s is lame. A flexible command line mp3 decoder and encoder.

Here we use 1:45 minutes 1.7MB music piece Lake Hylia from Zelda Reorchestrated. I don’t own any rights to this music and it’s used purely for demonstration purposes. I selected this piece because it’s melodic and instrumental like most game music is.

Key to reducing file size is lower sample and bitrate, and most importantly converting file to mono. Example below is how to do this with lame 3.99.5 on OS/X. I installed lame using Macports.

$ lame -hv -mm --resample 22.05 "Zelda Reorchestrated - Lake Hylia.original.mp3" -B 32 "Zelda Reorchestrated - Lake Hylia.small.mp3"
LAME 3.99.5 64bits (http://lame.sf.net)
Autoconverting from stereo to mono. Setting encoding to mono mode.
Resampling:  input 44.1 kHz  output 22.05 kHz
polyphase lowpass filter disabled
Encoding Zelda Reorchestrated - Lake Hylia.mp3
      to Zelda Reorchestrated - Lake Hylia.small.mp3
Encoding as 22.05 kHz single-ch MPEG-2 Layer III VBR(q=4)
    Frame          |  CPU time/estim | REAL time/estim | play/CPU |    ETA
  3996/3996  (100%)|    0:01/    0:01|    0:01/    0:01|   82.375x|    0:00
  8 [  36] **
 16 [  24] *
 24 [   4] *
 32 [3932] **************************************************************************************************************************************
-------------------------------------------------------------------------------------------------------------------------------------------------
   kbps       mono %     long switch short %
   31.7      100.0        99.9   0.1   0.1
Writing LAME Tag...done
ReplayGain: +3.7dB

This converts the input mp3 (lame also accepts .wav files) to 22.05kHz 64kbs monoaural mp3. The output file size is 412kB, resulting to nearly 75% savings. You can reduce file further by using even lower bitrate, (experiment -B 24 or -B 16), but you may start hearing distortions.

$ du -hs Zelda\ Reorchestrated\ -\ Lake\ Hylia.*
1.6M	Zelda Reorchestrated - Lake Hylia.original.mp3
404K	Zelda Reorchestrated - Lake Hylia.small.mp3

Test play the files here and listen for any differences.

Original: Zelda Reorchestrated – Lake Hylia.original.mp3
Reduced: Zelda Reorchestrated – Lake Hylia.small.mp3

MP3/WAV to reduced OGG Vorbis

This is main for Firefox as it still supports only Ogg Vorbis. You can convert mp3/wav file to reduced mono ogg vorbis with ffmpeg tool.

This example uses ffmpeg 2.1, and like lame I installed is on OS/X using Macports.

$ ffmpeg -i "Zelda Reorchestrated - Lake Hylia.original.mp3" -strict experimental -acodec libvorbis -ab 32k -ar 22050 -ac 1 "Zelda Reorchestrated - Lake Hylia.small.ogg"
ffmpeg version 2.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Oct 30 2013 23:24:10 with Apple LLVM version 5.0 (clang-500.2.79) (based on LLVM 3.3svn)
  configuration: --prefix=/opt/local --enable-swscale --enable-avfilter --enable-avresample --enable-libmp3lame --enable-libvorbis --enable-libopus --enable-libtheora --enable-libschroedinger --enable-libopenjpeg --enable-libmodplug --enable-libvpx --enable-libspeex --enable-libass --enable-libbluray --enable-gnutls --enable-libfreetype --disable-outdev=xv --mandir=/opt/local/share/man --enable-shared --enable-pthreads --cc=/usr/bin/clang --arch=x86_64 --enable-yasm --enable-gpl --enable-postproc --enable-libx264 --enable-libxvid --enable-nonfree --enable-libfaac
  libavutil      52. 48.100 / 52. 48.100
  libavcodec     55. 39.100 / 55. 39.100
  libavformat    55. 19.104 / 55. 19.104
  libavdevice    55.  5.100 / 55.  5.100
  libavfilter     3. 90.100 /  3. 90.100
  libavresample   1.  1.  0 /  1.  1.  0
  libswscale      2.  5.101 /  2.  5.101
  libswresample   0. 17.104 /  0. 17.104
  libpostproc    52.  3.100 / 52.  3.100
Input #0, mp3, from 'Zelda Reorchestrated - Lake Hylia.original.mp3':
  Metadata:
    title           : Lake Hylia
    artist          : Zelda Reorchestrated
    album           : Twilight Princess
    track           : 11
  Duration: 00:01:44.39, start: 0.000000, bitrate: 128 kb/s
    Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 128 kb/s
Output #0, ogg, to 'Zelda Reorchestrated - Lake Hylia.small.ogg':
  Metadata:
    title           : Lake Hylia
    artist          : Zelda Reorchestrated
    album           : Twilight Princess
    track           : 11
    encoder         : Lavf55.19.104
    Stream #0:0: Audio: vorbis (libvorbis), 22050 Hz, mono, fltp, 32 kb/s
    Metadata:
      title           : Lake Hylia
      artist          : Zelda Reorchestrated
      album           : Twilight Princess
      TRACKNUMBER     : 11
      encoder         : Lavf55.19.104
Stream mapping:
  Stream #0:0 -> #0:0 (mp3 -> libvorbis)
Press [q] to stop, [?] for help
size=     398kB time=00:01:44.38 bitrate=  31.2kbits/s
video:0kB audio:387kB subtitle:0 global headers:3kB muxing overhead 1.866299%

Resulting OGG file size and quality is comparable to reduced mp3.

$ du -hs Zelda\ Reorchestrated\ -\ Lake\ Hylia.*p3
400K	Zelda Reorchestrated - Lake Hylia.small.ogg
1.6M	Zelda Reorchestrated - Lake Hylia.original.mp3

Compare here:

Original: Zelda Reorchestrated – Lake Hylia.original.mp3
Reduced: Zelda Reorchestrated – Lake Hylia.small.ogg

Simple Slot machine game using HTML5 Part 4: Offline mode

This is the fourth part of the Slot machine game in HTML5 (previous parts 1, 2, and 3) and this time we modify the game to support HTML5 offline mode, also known as HTML5 Application Cache.

Slots Offline

Try out offline supported version here.

Word of warning. HTML5 offline mode is powerful but very fragile feature. It’s tricky to get right, but once you get it to work the mobile user experience can be very native app like.

Some problems you will encounter

  • Browser refresh logic is confusing. Especially the fact that browser does not use updated manifest and resources when they change but only after next reload. Fortunately Javascript workarounds exists.
  • Application cache file maintenance needs diligence. For example, browser will not reload any assets if this file is not modified.
  • Externally linked resources do not generally work offline. This makes CDN use difficult.
  • Web server has to use right MIME type and cache settings to reliably use application cache files. Most web servers don’t do this in default configuration.
  • No reliable way to detect if page was loaded in online or offline mode.
  • Chrome bypasses some restrictions (e.g. cross-domain issues) in the specification and what works in Chrome may not work anywhere else.
  • Each browser has slightly different meaning and heuristic for using offline mode. For example if browser can load some unrelated pages but can’t currently load your app page it may not show your page in offline mode and simply shows “Can not reach the server error”. This may happen especially if it knows from last load that manifest has been updated. Then at other times, it may load page few times in offline mode even when connectivity has returned.

Manifest file

Web page must use application cache manifest file to support offline mode. This manifest file is specified in html tag of the page.

<!DOCTYPE html>
<html manifest="slots.appcache">
<head>
 ...

The file has listing of all content that page needs. For detailed explanation of each section, refer to Beginners guide to HTML5 application cache

CACHE MANIFEST
# version 10

NETWORK:
# here goes resources that must be never cached
js/online.json

CACHE:
audio/nowin.mp3
audio/nowin.ogg
audio/roll.mp3
audio/roll.ogg
audio/slot.mp3
audio/slot.ogg
audio/win.mp3
audio/win.ogg
css/webfonts.css
css/reset.css
css/slot.css
css/Slackey.ttf
img/build-64.png
img/cash-64.png
img/energy-64.png
img/gold-64.png
img/goods-64.png
img/loading.gif
img/staff-64.png
js/slot.js
js/jquery.min.js
index.html

Web Fonts

Web fonts must be hosted locally if you want to use them offline. Note that some web fonts may have licensing restrictions for local hosting.

<link type='text/css' rel='stylesheet' href="css/webfonts.css"/>

The webfont.css defines the font face and loads true type file.

@font-face {
  font-family: 'Slackey';
  font-style: normal;
  font-weight: 400;
  src: local('Slackey'), url(Slackey.ttf) format('truetype');
}

The file Slackey.ttf is hosted locally in css directory.

Web Server Support

Web server must use correct MIME type for text/cache-manifest application cache manifest. For example, in NGINX web server edit the mime.types and add following file type to MIME type mapping.

text/cache-manifest		appcache;

Browsers should check manifest every time page is loaded online, but it may not do this often enough if cache control is too long. Therefore, set short cache lifetime for the manifest files by adding this inside server section of NGINX configuration file. This forces cache lifetime of 1 minute to all *.appcache files.

# set 1 minute cache life for HTML5 offline manifests
location ~* \.(appcache)$ {
   expires 1m;
}

Verify with cURL that server response Content-Type has right MIME type and that the Expires and/or Cache-Control have correct 1 minute cache life time. If you get 404 error, make sure that site root configuration is set in http section.

$ curl -I http://localhost:8081/slots.appcache
HTTP/1.1 200 OK
Server: nginx/1.4.0
Date: Sun, 05 May 2013 04:43:32 GMT
Content-Type: text/cache-manifest
Content-Length: 38
Last-Modified: Sun, 05 May 2013 04:25:28 GMT
Connection: keep-alive
ETag: "5185df38-26"
Expires: Sun, 05 May 2013 04:44:32 GMT
Cache-Control: max-age=60
Accept-Ranges: bytes

Detecting online status

Currently only “reliable” method to do is to make Ajax request and check the response. There are some caveats

  • Request may fail for other reasons, and this does not mean browser is in offline mode
  • Offline status may change while user is in page, you may want to do repeat polling check.
  • User may be in public WiFi that redirects requests to login server. This can confuse your app that gets response but is not what was expected.

Slots game checks online status in parallel while game loads and only on startup. Slots game does not really need to know if it’s online or offline, but just writes the status on screen for debugging purposes.

<script type="text/javascript">$(function () {

    var game = SlotGame();

    // Attempt loading static json file from server to detect online or offline mode.
    // The url has unique random parameter to avoid browser or proxy caches
    $.ajax({
        url: 'js/online.json?ts=' + (~~new Date()),
        dataType: 'json',
        success: function(data) { 
            if ( data.online ) { 
                game.setOnlineStatus(true);
            } else {
                // might be online, but we didn't get expected response. Could be
                // e.g. Wifi login page.
                game.setOnlineStatus(false);
            }
        },
        error: function() {
            game.setOnlineStatus(false);
        }
    });

});</script>

Otherwise loading images, audio and other content should be fully transparent to your app. Think twice before doing separate logic for online and offline as things will get difficult. Best advice I can give is to that you write code for online use with proper handling for Ajax errors. In this way when app loads in online mode but loses network later in session (e.g. when user goes in subway tunnel), the experience does not break completely.

Testing

This is the part where things get interesting, offline is tricky to test because of caching and browser reload logic. See detailed lamentation about subject here in Dive into HTML5.

These are the best practices I’ve come up with. First, set browser manually to offline mode to try things out. e.g. in Firefox this is enabled from File->Work Offline.

Firefox offline

Second, if you develop the game from local server, do not use http://localhost as host, but use real domain name that resolves to localhost. In this example I’ve used http://hexxie.com that supports wildcard subdomain. Any subdomain resolves to address 127.0.0.1.

$ nslookup anything-goes-here.hexxie.com
Server:		192.168.1.254
Address:	192.168.1.254#53

Non-authoritative answer:
Name:	anything-goes-here.hexxie.com
Address: 127.0.0.1

In this way you can always start from scratch the offline debugging simply by changing subdomain name. For example I just used slotsoff1.hexxie.com, slotsoff2.hexxie.com, … etc.:

Trick url

Note that at least Firefox asks each time if you allow offline content.
Accept offline

Before each deploy, remember to increment the version comment in manifest file, so web server notices that the file has changed and browser will refresh it on next load. Server does not look inside the manifest file, so it does not matter how you change the file, as long as it’s changed.

Good Luck!

Code is available in Github.