C++11 Variadic Templates implementation of parameterized strings

C++11 introduces variadic templates that allow nice way to have generic functions with unknown arguments, especially when used in recursive way. This makes easy to implement functions that before were cumbersome to write or hard to use.

Nice use case are parametric formatted strings where part of the strings are replaced with run time data. For example:

"Hello {0}, how are you?"
"Hello {name}, you logged in last {date}".

This find of format is quite easy to implement with recursive variadic templates so that it supports both indexed  and named replacement.

inline std::string format_r(int /*pos*/, std::string format) { return format; }
inline std::string format(const std::string format) { return format; }

template <typename T, typename... ARGS>
std::string format_r(int pos, std::string format, const T&amp; value, ARGS... args);

// convenience short hand for building a format parameter
template <typename... ARGS>
std::pair<ARGS...> _p(ARGS... args) { return std::make_pair(std::forward<ARGS>(args)...); };

template <typename K, typename T, typename... ARGS>
std::string format_r(int pos, std::string format, const std::pair<K, T>& value, ARGS... args)
    std::ostringstream os;
    os << value.second;
    auto parameter = str("{", value.first, "}");
    replace_all(format, parameter, std::string(os.str()));
    return format_r(pos + 1, format, std::forward<ARGS>(args)...);

template <typename T, typename... ARGS>
std::string format_r(int pos, std::string format, const &T value, ARGS... args)
    return format_r(pos, format, std::make_pair(str(pos), value), std::forward<ARGS>(args)...);

template <typename T, typename... ARGS>
std::string format(const std::string format, const &T value, ARGS... args)
    return format_r(0, format, value, std::forward<ARGS>(args)...);


Format function wraps a recursive function that builds the string by replacing one parameter at a time and then processing it again for the next argument until it runs out of parameters.

String conversions are handled with std::ostringstream that supports overloaded ‘<<‘ operation for most common data types.

Format function can be called with index formatting:

using util::format;
format("Hello {0}!", "World"); // Hello World
format("{0} {1}!", "Hello", "World"); // Hello World
format("{0} + {1} = {2}", 1, 2, 3); // 1 + 2 = 3

Or alternatively with explicit parameter names:

using util;
format("Hello {what}!", _p("what", "World"));
format("{word} {other}!", _p("word", "World"), _p("other", "World"));
format("{a} + {b} = {c}", _p("a", 1), _p("b", 2), _p("c", 3));

Get full implementation ftom https://github.com/tikonen/blog/tree/master/variadicformat

Update Gandi.net DNS on Amazon EC2 server boot up

Cloud servers get often randomly picked public IP when they are booted up.  Here is a script can be used to update the Gandi.net DNS records automatically on EC2 server startup.

Step 1. Get API key

Activate and get your Gandi.net API key with  these instructions.  Also, ensure that your Zone file for the domain is updatable. Usually this means that you’ve made copy of the template Zone file.

Step 2. Configure

Download update_domain.py python script and fill in your domain details and the Gandi.net API key.

DOMAIN = "yourdomain.com" # the domain name to update
NAMES  = ["@", "www"]     # A record names to update
API_KEY= '*********'      # fill in gandi API key

Configuration above would update IP for records ‘yourdomain.com’ and ‘www.yourdomain.com’.

You can also redefine the function ‘resolve_ip’ to adapt the script for other environments than EC2. Current implementation uses EC2’s internal REST API to query instances public IP.

Step 3. Run and test

Run the script on the EC2 server, it should resolve the local IP, Zone file and check if records need to be updated.

$ python update_domain.py

Script does dry run by default and will not update records, set DRY_RUN flag to False to update the records for real.

DRY_RUN = False          # Set false to actually modify Zone

Step 4. Run on boot up

When you’re satisfied with the settings and tested script manually, run command ‘crontab -e’ and add the following entry.

@reboot python /home/ubuntu/update_domain.py

Cron will now run the script on every reboot.

Quick and robust C++ CSV reader with boost

This is quick and simple CSV reader based on Boost regular expression token iterator. Parser splits the input with a regular expressions and returns the result as a collection of vectors of strings.
Regular expression handles neatly lot of the complicated edge cases such as empty columns, quoted text, etc..

Parser code

#include <boost/regex.hpp>

// used to split the file in lines
const boost::regex linesregx("\\r\\n|\\n\\r|\\n|\\r");

// used to split each line to tokens, assuming ',' as column separator
const boost::regex fieldsregx(",(?=(?:[^\"]*\"[^\"]*\")*(?![^\"]*\"))");

typedef std::vector<std::string> Row;

std::vector<Row> parse(const char* data, unsigned int length)
    std::vector<Row> result;

    // iterator splits data to lines
    boost::cregex_token_iterator li(data, data + length, linesregx, -1);
    boost::cregex_token_iterator end;

    while (li != end) {
        std::string line = li->str();

        // Split line to tokens
        boost::sregex_token_iterator ti(line.begin(), line.end(), fieldsregx, -1);
        boost::sregex_token_iterator end2;

        std::vector<std::string> row;
        while (ti != end2) {
            std::string token = ti->str();
        if (line.back() == ',') {
            // last character was a separator
    return result;


CSV data with common problem cases, such as empty quotes, commas inside quotes and empty last column.

3,a b,5
7,x,long story no commas
8,y,"some, commas, here,"

Read and parse the CSV data above and output the parsed result

int main(int argc, char*argv[])
	// read example file
	std::ifstream infile;
	char buffer[1024];
	infile.read(buffer, sizeof(buffer));
	buffer[infile.tellg()] = '\0';

	// parse file
	std::vector<Row> result  = parse(buffer, strlen(buffer));

	// print out result
	for(size_t r=0; r < result.size(); r++) {
		Row& row = result[r];
		for(size_t c=0; c < row.size() - 1; c++) {
			std::cout << row[c] << "\t";
		std::cout << row.back() << std::endl;


$ ./reader
a      	b      	c
1      	"cat"  	3
",2"   	dog    	4
3      	a b    	5
4      	empty
5      	        empty
6      	""      empty2
7      	x      	long story no commas
8      	y      	"some, commas, here,"

See full example code in Github: https://github.com/tikonen/blog/tree/master/boostcsvreader

Unity Debug.Log with multiple arguments

Javascript has neat debugging function console.log that accepts multiple variables which makes easy to compose and modify debug output. It’s easy to do same kind of utility script for the Unity.

public class Console
    public static void Log(params object[] a)
        var s =a[0].ToString();
        for ( int i = 1; i < a.Length; i++ ) {
            s += " ";
            s += a[i].ToString();

Now it’s easy to write debug strings like this

var i = 4;
var a = "the";
Console.Log("Hello", i, a, "World");
// => "Hello 4 the World"

Instead of this crappy string concatenation..

var i = 4;
var a = "the";
Debug.Log("Hello " + i.ToString() + " " + a + "World");
// "Hello 4 theWorld" .. forgot one space :(

Quickstart for SQLite in Unity

I had some troubles setting up SQLite for my Unity project for Android and iOS app. Let’s hope this quickstart helps you faster up to speed with your mobile app.

DISCLAIMER. This example is for OS/X development environment and Android and iOS builds. Never tried this for Windows but I guess installing sqlite3.dll should do the trick.

Step 1. Get wrapper API

First, get the SQLite library by @busta117 from Github: https://github.com/Busta117/SQLiteUnityKit

Step 2. Copy API files on your project

Copy the DataTable.cs and SqliteDatabase.cs somewhere under your projects Assets/Scripts/ folder. If you build also for android, then copy libsqlite3.so in your projects Assets/Plugins/Android/ folder. iOS does not need plugin as it has native support for sqlite.

Step 3. Create default database

This is the database that should contain the tables you need with any default data you may want to have. Default database is used to bootstrap the actual in app database.

Create folder Assets/StreamingAssets/. Then create your default template database with sqlite3.

$ sqlite3 Assets/StreamingAssets/default.db
SQLite version 2015-02-25 13:29:11
Enter ".help" for usage hints.
sqlite> create table example (
   ...> name string,
   ...> dummy int
   ...> );
sqlite> .schema example
CREATE TABLE example (
name string,
dummy int
sqlite> insert into example values ("hello world", 1);
sqlite> select * from example;
hello world|1

Now you have default database and your project should have files like these.


Step 4. Database initialization code.

Initialize the database in your main scripts Awake() method. This checks if database already exists and if not, it copies the default db as template.

SqliteDatabase sqlDB;

void Awake() 
    string dbPath = System.IO.Path.Combine (Application.persistentDataPath, "game.db");
    var dbTemplatePath = System.IO.Path.Combine(Application.streamingAssetsPath, "default.db");

    if (!System.IO.File.Exists(dbPath)) {
        // game database does not exists, copy default db as template
        if (Application.platform == RuntimePlatform.Android)
            // Must use WWW for streaming asset
            WWW reader = new WWW(dbTemplatePath);
            while ( !reader.isDone) {}
            System.IO.File.WriteAllBytes(dbPath, reader.bytes);
        } else {
            System.IO.File.Copy(dbTemplatePath, dbPath, true);
    sqlDB = new SqliteDatabase(dbPath);

You can use the script execution order setting to ensure that this code is always executed first.

Step 5. Use the database!

API supports normal selects, inserts and updates.

result = sqlDB.ExecuteQuery("SELECT * FROM example");
row = result.Rows[0];
print("name=" + (string)row["name"]);
print("dummy=" + (int)row["dummy"]);

API is simple to use, check detailed documentation from the https://github.com/Busta117/SQLiteUnityKit.

Finding maximal rectangles in a grid

Maximal rectangles problem is to find rectangles inside an area that have maximum possible surface area. This is often needed in games to show the player where in area a certain item can fit etc.. There are many ways to approach the problem, and for some cases (such as areas defined by polygon) they can get complex to implement. Here is one relatively simple method for looking up those maximum rectangles inside an area in a common grid. It’s pretty efficient and does support following restrictions

  • Rectangles can have minimum size (e.g. larger than 2×2).
  • There can be more than one maximum rectangle in the area. (in case two large areas are joined with narrow path)

First part is defining function that looks up rectangle dimensions. This function starts from a tile that will be left top corner of rectangle and returns its width and height. The listings here are in pseudocode.

Listing 1. Get rectangle size

FUNC get_rectangle_size( tile, tiles )
    # find rectangle width
    width = 1
    WHILE contains( (tile.x + width, c.y,  tiles )
      width = width + 1
    # find rectangle height
    height = 0
    stop = false
       height = height + 1
       LOOP FROM x = tile.x TO tile.x + width
         IF NOT contains( (x, tile.y + height), tiles )
            stop = true
            x = x + 1
    UNTIL stop  

    RETURN (width, height)

Function finds the maximum width of the rectangle starting from the left corner and working right (lines 5-6), then it finds rectangle height by looking up rows of same width (lines 11-20).

The actual algorithm that finds the rectangles accepts list of grid tiles as input and outputs the found rectangles that are maximally sized. The pseudocode listing here assumes some helper functions for simplicity.

  • sort ( list, compare ) – Sorts list in place by compare function
  • add( item, list ) – Adds item to a list
  • remove (item, list) – Removes item from list
  • contains( item, list ) – Returns true if item is found from list
  • length( list ) – Returns lists length
  • overlap (rect1, rect2) – Returns true if two rectangles overlap

Listing 2. Find maximal rectangles. Unoptimized for readability!

FUNC find_tiles( tiles )  # the list of tiles as (x,y) tuples
  rectangles = []  # list of found rectangles

  # sort with compare order by y and x.
  sort(tiles,  (a, b) => a.y == b.y ? a.x - b.x : a.y - b.y)

  WHILE length(tiles) > 0
    c = head(tiles)   # take first item from the sorted list
    # get rectangle size
    (width, height) = get_rectangle_size( c, tiles ) 

    # remove tiles that can not be part of any larger rectangle
    LOOP FROM x = c.x TO c.x + width
      LOOP FROM y = c.y TO c.y + height
         IF NOT contains( (c.x - 1, y, tiles) AND
            NOT contains( (c.x + width, y), tiles) AND
            NOT contains( (x, c.y - 1), tiles) AND
            NOT contains( (x, c.y + height), tiles)
            remove( (x, y), tiles )
    # if big enough rectangle add it to list
    IF width > 1 AND height > 1
      add ( (c.x, c.y, width, height), rectangles ) 

  # sort rectangles by area size
  sort( rectangles, (r1, r2) => r2.width * r2.height - r1.width * r1.height )

  # remove overlapping rectangles
  i = 0
  WHILE i < length(rectangles) - 1
    LOOP FROM j = length(rectangles) - 1 TO i + 1 # descending order 
        IF overlaps(rectangles[i], rectangles[j])
          remove(rectangles[j], rectangles)
    i = i + 1

  RETURN rectangles

Here is example how the algorithm works, here white area presents the working set of tiles that algorithm works on, white presents the tiles that are in list tiles.

Algorithm sorts first the tiles in top-to-bottom and left-to-right order so it starts from top left tile. The pink denotes the current working tile c. The green represents the tiles that belong to rectangle as determined by call to get_rectangle_size.

Next step is to remove tiles from working set that can not be part of any other rectangle. This is determined by checking if the row and column of tile are bounded inside the current rectangle. The purple presents tiles that were removed from the working set tiles. Found rectangle is added in the list rectangles if it’s large enough (in this case larger than 2×2).
Then loop is executed again with next corner tile c.

Next iteration of the main loop returns another rectangle and closed tiles are removed and rectangle added to list.

Final loop of the rectangle. There are no more tiles to work so main loop stops.

Finally algorithm sorts the found rectangles in descending surface area order and removes overlap. The resulting rectangles are returned as output.

Class Persistence in Unity

Games need to store some persistent data like high scores and progress between the game sessions. Fortunately Unity gives us PlayerPrefs class that is essentially a persistent hash map.

Reading and writing values with PlayerPrefs is as simple as calling get and set.

// saving data
PlayerPrefs.SetInt("foobar", 10);
PlayerPrefs.SetString("something", "foo");
// reading data
if(PlayerPrefs.HasKey("foobar")) {
    int foo = PlayerPrefs.GetInt("foobar");

It has its limitations, only strings and numbers can be stored and that makes more complex data lot more difficult to maintain.

What we can do is to write simple utility that can be used to serialize classes to strings that can then be read and written with PlayerPrefs. SerializerUtil is static class with two methods to Load and Write object. In case loading fails it returns default value of the data, usually null.

using System;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Formatters.Binary;

public static class SerializerUtil {
	static BinaryFormatter bf = new BinaryFormatter ();
	public static T LoadObject<T>(string key)
		if (!PlayerPrefs.HasKey(key))
			return default(T);
		try {	
			string tmp = PlayerPrefs.GetString(key);
			MemoryStream dataStream = new MemoryStream(Convert.FromBase64String(tmp));
			return (T)bf.Deserialize(dataStream);
		} catch (Exception e) {
			Debug.Log("Failed to read "+ key+ " err:" + e.Message);
			return default(T);
	public static void SaveObject<T>(string key, T dataObject)
		MemoryStream memoryStream = new MemoryStream ();
		bf.Serialize (memoryStream, dataObject);
		string tmp = Convert.ToBase64String (memoryStream.ToArray ());
		PlayerPrefs.SetString ( key, tmp);

To load and save your classes, declare them as Serializable.

public class PlayerData {
	public int points;
	public string name; 		

You can then easily store instances of the class.

var data = new PlayerData();
data.points = 50;
data.name = "Teemu";

SerializerUtil.SaveObject("player1", data);

Reading classes is also simple

PlayerData data;
data = SerializerUtil.LoadObject<PlayerData>("player1");
// data.points is 50
// data.name is "Teemu"

Classes can be also more complicated, you can add member functions and also exclude some member variables from serialization. It’s also possible to define member function that will be called after serialization, it’s good way to init class after deserialization from disk.

public class PlayerData : IDeserializationCallback {
	public int points;
	public string name;
	// serialization ignores this member variable
	[NonSerialized] Dictionary<int, int> progress = new Dictionary<int, int>();

	// constructor is called only on new instances
	public PlayerData() {
		points = 0;
		name = "Anon";
	// serializable class can have member methods as usual
	public bool ValidName() 
		return name.Trim().Length > 4; 

	// Called only on deserialized classes
	void IDeserializationCallback.OnDeserialization(System.Object sender) 
	    // do your init stuff here
	    progress = new Dictionary<int, int>();	


This serialization does not support versioning. You can not read the stored instance anymore if you change the class members as the LoadObject will fail to deserialize the data.

You need to add following environment variable to force Mono runtime to use reflection instead JIT. Otherwise the serialization will fail on iOS devices. Do this before doing any loading or saving of classes.

void Awake() {
    Environment.SetEnvironmentVariable("MONO_REFLECTION_SERIALIZER", "yes");