Polling Apple Push Notification feedback service with Node.js

Apple assumes that your app polls the Push Notification Feedback Service to get information about App uninstalls so you can stop sending  notifications to those devices. Service returns list of tokens with uninstall timestamps.

This is how you can do it with Node.js. First, check TLS Example and how to create SSL certs to understand basics of how to make secure SSL connection with Node.js.

First simple utility to convert binary data to hex

function bintohex(buf) {

    var hexbuf = new Buffer(buf.length * 2);

    function nibble(b) {
	if (b <= 0x09) return 0x30 + b;
	return 0x41 + (b - 10);
    }

    for(var i=0; i < buf.length; i++) {
        hexbuf[i*2] = nibble(buf[i] >> 4);
        hexbuf[i*2+1] = nibble(buf[i] & 0x0F);
    }
    return hexbuf.toString('ascii', 0, hexbuf.length);
}

Then define function that will be called with push device tokens

function processTokens( pdtokens ) {

    // send or process tokens here
    pdtokens.forEach( function( pd  ){
	console.log('TOKEN ' + pd.token ' INVALIDATED AT ' + new Date( pd.timestamp ) );
    });
}

Then polling function that does it all, note that it assumes function connectAPN that needs simply create SSL connection to ‘feedback.push.apple.com’ port 2196 with your apps client cert and private key.

function pollAPNFeedback() {

    console.log("Connecting APN feedback service");

    connectAPN(function( apnc ) {

	var bufferlist = [];
	apnc.on('data', function(data) {
	    // APN feedback starts sending data immediately on successful connect
	    bufferlist.push(data)
	});

	apnc.on('end', function() {
	    console.log("APN Connection closed");

	    // tokens are parsed to object
	    var pdtokens = []

	    // concatenate all asynchronously collected buffers, and parse the PDU's from them
	    var parsebuf;
	    var totall = 0;
	    for(var i=0; i < bufferlist.length; i++) {
	        totall += bufferlist[i].length;
	    }
	    parsebuf = new Buffer(totall);
	    var offset = 0;
	    for(var i=0; i < bufferlist.length; i++) {
		bufferlist[i].copy(parsebuf, offset, 0);
		offset += bufferlist[i].length;
	    }

	    var count = 0;
	    for(var k = 0; k < parsebuf.length; ) {
		count++;
                 // parse timestamp
		var ts = ((parsebuf[k] << 24) + (parsebuf[k+1] << 16) + (parsebuf[k+2] << 8) + parsebuf[k+3]) >>> 0;
		k += 4;
		var l = ((parsebuf[k] << 8) + parsebuf[k]) >>> 0;
                  k += 2;
                  var pdtokenbuf = new Buffer(l);
		parsebuf.copy(pdtokenbuf, 0, k, k + l);
		k += l;

		var hextoken = bintohex(pdtokenbuf);
		pdtokens.push({timestamp: ts, token:hextoken});

		// process tokens in 100 item batches
	        if( count >= 100 ) {
		    processTokens( pdtokens );
		    count = 0;
		    pdtokens = [];
		}
	}
	processTokens(pdtokens);
         setTimeout(pollAPNFeedback, 1000*60*60*4);
    });
}

pollAPNFeedback();

The function read raw binary data from connection and after Feedback services closes the socket (this happens immediately after it has sent the data) it parses tokens and calls processing function to handle them. Function connects to server every 4 hours.

Note that feedback service host and port is different for sandbox (testing environment) that also needs its own SSL certificates.

Simple popularity algorithm with decay

Most social web sites have user created objects such as pictures, posts, links, etc.. that need to be shown in some dynamic popularity or ranking order on website. The measurement of popularity is typically some user activity such as votes, downvotes, number of clicks or shares per object.

It would be trivial just to calculate total sum of the actions, and sort objects by the sum but this doesn’t take into account the bandwagon effect where people start voting the already top items so the top list freezes quickly and no new items have change to bubble on top. Another problem of this approach is that once object reaches high score it stays high even when no actions would anymore happen. Good popularity measurement needs to decay somehow.

First page of top list (typically 10-20 items)  should always show fresh “hot” items together with long time favorites. We want that older objects fade and need more and more actions to stay on the top list as time passes by.

Simple way to do this is with relative scoring, where base score  grows constantly so that score for younger object is always naturally higher than for old objects with same number of actions. Object creation timestamp can be used trivially as base score. For example:

object_score = created_timestamp + number_of_actions

created_timestamp is object creation time as number of hours or days from some arbitrary fixed reference time . The popularity list is simply all objects sorted by the score. New objects have naturally higher score and old ones need more and more actions to stay on top.

Example in Javascript

Fixed reference time 1st Jan 2000.  Timestamp as hours

Object A created on midnight 1st May 2012, 30 votes

(new Date('2012-5-1') - new Date('2000-1-1'))/1000/60/60 + 30 = 108126

Object B created on midnight 2nd May 2012 2 votes

(new Date('2012-5-2') - new Date('2000-1-1'))/1000/60/60 + 2 = 108122

So, on 2nd May the object A score would still be ahead B but the B needs only 5 more votes to pass A. The A will fall behind quickly unless it gets constantly more votes. In fact with this trivial formula object has to get 24 actions per day just to keep up with the increasing base score.

Major benefit of growing base score is that its one pass, i.e. app  updates score for every new action and thats it. There is no need to do computation on query and no periodic maintenance is required. It’s also compatible with Map-Reduce based views (e.g. CouchDB) as its not dependent of query time.

I assume many high traffic web sites (Digg for sure) use  some derivative of this basic method, but each site needs to tune it based on their actions, traffic and user behavior.

See also Simple Cooldown and Rate Control Algorithm for another perspective to time based scoring.