Test server for Apple push notification and feedback integration

Here is a test server that you can use to verify your integration to Apple Push notification service and push notification feedback server. It should be good enough for testing out your application behavior and pdu format. Server helps you to get more debug info than just connecting directly to apple.

See these blog posts on details how to send push notifications

Test server runs under node.js and listens both SSL and plain ports where your application can connect. Get code from Github and generate the SSL keys (see quick howto in cert-howto.txt) and start up server

$ node server.js 
Waiting for connections in ports
Listening feedback port 2296 
Listening feedback port 2196 SSL
Listening push port 2295 
Listening push port 2195 SSL

Note that you may need to install binary module to run the server. Use npm install binary.

Successful push notification sending should look like following. The server dumps the data from your app in binary format for debugging and tries to parse it as Push PDU. Server prints out the fields so you can verify the data.

Accepted push connection 2195 1 SSL
=== RECEIVED DATA (1) ====
00000000: 0100 0000 0151 7261 1d00 206b 4628 de93  .....Qra...kF(^.
00000010: 17c8 0edd 1c79 1640 b58f dfc4 6d21 d0d2  .H.].y.@5._Dm!PR
00000020: d135 1687 239c 44d8 e30a b100 1e7b 2261  Q5..#.DXc.1..{"a
00000030: 7073 223a 7b22 616c 6572 7422 3a22 4865  ps":{"alert":"He
00000040: 6c6c 6f20 5075 7368 227d 7d              llo.Push"}}

=== PDU ====
{ command: 1,
  pduid: 1,
  expiry: Sat Apr 20 2013 17:34:21 GMT+0800 (SGT),
  tokenlength: 32,
  token: '6B4628DE9317C80EDD1C791640B58FDFC46D21D0D2D1351687239C44D8E30AB1',
  payloadlength: 30,
  payload: { aps: { alert: 'Hello Push' } } }

Test server does not validate the data, but it tries to parse JSON message in push notifications and prints error if it fails. Also if command was not set to 1, it sends back error pdu and closes connection. This should be good enough for testing. For example here I made HTTP request to the server to get some error output.

Accepted push connection 2295 1 SSL
=== RECEIVED DATA (1) ====
00000000: 4745 5420 2f20 4854 5450 2f31 2e31 0d0a  GET./.HTTP/1.1..
00000010: 5573 6572 2d41 6765 6e74 3a20 6375 726c  User-Agent:.curl
00000020: 2f37 2e32 392e 300d 0a48 6f73 743a 206c  /7.29.0..Host:.l
00000030: 6f63 616c 686f 7374 3a32 3239 350d 0a41  ocalhost:2295..A
00000040: 6363 6570 743a 202a 2f2a 0d0a 0d0a       ccept:.*/*....

=== PDU ====
{ command: 71,
  pduid: 1163141167,
  expiry: Sun Mar 01 1987 23:31:32 GMT+0800 (SGT),
  tokenlength: 20527,
  token: '312E310D0A557365722D4167656E743A206375726C2F372E32392E300D0A486F73743A206C6F63616C686F73743A323239350D0A4163636570743A202A2F2A0D0A0D0A',
  payloadlength: null,
  payload: 'ERROR: INVALID JSON PAYLOAD [SyntaxError: Unexpected end of input]' }
=== SEND ERROR: 08014554202F
Connection terminated 1

When your app connects successfully to the feedback test service, it sends back few feedback tokens and closes connection after one minute. Edit the actual tokens in the server source code.

Accepted feedback connection 2296 1
SEND: 2696A21000207518B1C2C7686D3B5DCAC8232313D5D0047CF0DC0ED5D753C017FFB64AD25B60
SEND: 2696A21100207518B1C2C7686D3B5DCAC8232313D5D0047CF0DC0ED5D753C017FFB64AD25B60
SEND: 2696A21100207518B1C2C7686D3B5DCAC8232313D5D0047CF0DC0ED5D753C017FFB64AD25B60

Source code is available in Github.

Polling Apple Push Notification feedback service with Node.js

Apple assumes that your app polls the Push Notification Feedback Service to get information about App uninstalls so you can stop sending  notifications to those devices. Service returns list of tokens with uninstall timestamps.

This is how you can do it with Node.js. First, check TLS Example and how to create SSL certs to understand basics of how to make secure SSL connection with Node.js.

First simple utility to convert binary data to hex

function bintohex(buf) {

    var hexbuf = new Buffer(buf.length * 2);

    function nibble(b) {
	if (b <= 0x09) return 0x30 + b;
	return 0x41 + (b - 10);
    }

    for(var i=0; i < buf.length; i++) {
        hexbuf[i*2] = nibble(buf[i] >> 4);
        hexbuf[i*2+1] = nibble(buf[i] & 0x0F);
    }
    return hexbuf.toString('ascii', 0, hexbuf.length);
}

Then define function that will be called with push device tokens

function processTokens( pdtokens ) {

    // send or process tokens here
    pdtokens.forEach( function( pd  ){
	console.log('TOKEN ' + pd.token ' INVALIDATED AT ' + new Date( pd.timestamp ) );
    });
}

Then polling function that does it all, note that it assumes function connectAPN that needs simply create SSL connection to ‘feedback.push.apple.com’ port 2196 with your apps client cert and private key.

function pollAPNFeedback() {

    console.log("Connecting APN feedback service");

    connectAPN(function( apnc ) {

	var bufferlist = [];
	apnc.on('data', function(data) {
	    // APN feedback starts sending data immediately on successful connect
	    bufferlist.push(data)
	});

	apnc.on('end', function() {
	    console.log("APN Connection closed");

	    // tokens are parsed to object
	    var pdtokens = []

	    // concatenate all asynchronously collected buffers, and parse the PDU's from them
	    var parsebuf;
	    var totall = 0;
	    for(var i=0; i < bufferlist.length; i++) {
	        totall += bufferlist[i].length;
	    }
	    parsebuf = new Buffer(totall);
	    var offset = 0;
	    for(var i=0; i < bufferlist.length; i++) {
		bufferlist[i].copy(parsebuf, offset, 0);
		offset += bufferlist[i].length;
	    }

	    var count = 0;
	    for(var k = 0; k < parsebuf.length; ) {
		count++;
                 // parse timestamp
		var ts = ((parsebuf[k] << 24) + (parsebuf[k+1] << 16) + (parsebuf[k+2] << 8) + parsebuf[k+3]) >>> 0;
		k += 4;
		var l = ((parsebuf[k] << 8) + parsebuf[k]) >>> 0;
                  k += 2;
                  var pdtokenbuf = new Buffer(l);
		parsebuf.copy(pdtokenbuf, 0, k, k + l);
		k += l;

		var hextoken = bintohex(pdtokenbuf);
		pdtokens.push({timestamp: ts, token:hextoken});

		// process tokens in 100 item batches
	        if( count >= 100 ) {
		    processTokens( pdtokens );
		    count = 0;
		    pdtokens = [];
		}
	}
	processTokens(pdtokens);
         setTimeout(pollAPNFeedback, 1000*60*60*4);
    });
}

pollAPNFeedback();

The function read raw binary data from connection and after Feedback services closes the socket (this happens immediately after it has sent the data) it parses tokens and calls processing function to handle them. Function connects to server every 4 hours.

Note that feedback service host and port is different for sandbox (testing environment) that also needs its own SSL certificates.

Apple Push Notifications with Erlang

Continuing from the Node.js based example I wrote earlier, here is example how to do the same with Erlang. You can check more details from the previous post, but as reminder the Apple Push Notification interface is simple binary based protocol that you use over SSL authenticated socket.

1. Prerequisites

I assume you have erlang installed, the version I’m using here is Erlang R13B03 (erts-5.7.4).

Check instructions here at Node.js based example how to get the push certificates as .pem files.

Install mochiweb package in your erlang environment.

Check that you’ve all set.

$ ERL_LIBS=. erl
Erlang R13B03 (erts-5.7.4)  [64-bit] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
2> mochijson:encode("cat").
"\"cat\""
3> application:start(ssl).
ok

Later releases of Erlang may require you to start ‘crypto’ and ‘public_key’ applications before starting ssl.

2. Sending Push Notification

First code to convert hexadecimal strings to binary format. This is mainly for readability for the example. I don’t remember where I snacked that code, but it seems to be found from several sites around the Intertubes.

-module(hex).
-export([bin_to_hexstr/1,hexstr_to_bin/1]).

bin_to_hexstr(Bin) ->
   lists:flatten([io_lib:format("~2.16.0B", [X]) ||
                  X <- binary_to_list(Bin)]).

hexstr_to_bin(S) ->
   hexstr_to_bin(S, []).
hexstr_to_bin([], Acc) ->
   list_to_binary(lists:reverse(Acc));
hexstr_to_bin([X,Y|T], Acc) ->
   {ok, [V], []} = io_lib:fread("~16u", [X,Y]),
   hexstr_to_bin(T, [V | Acc]).

Then the code to actually connect to APN and send the PDU

-module(ssltest).
-export([sendpush/0]).
-import(hex).

sendpush() ->
  Address = "gateway.sandbox.push.apple.com",
  Port = 2195,
  Cert = "cert.pem",
  Key = "key-noenc.pem",  

  %Options = [{cacertfile, CaCert}, {certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Options = [{certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Timeout = 1000,
  {ok, Socket} = ssl:connect(Address, Port, Options, Timeout),

Open SSL socket to the APN server with application certificate and private key.

  Payload = mochijson:encode({struct, [{"aps", {struct, [{"alert", "This is Message"}]}}]}),
  BPayload = erlang:list_to_binary(Payload),
  PayloadLen = erlang:byte_size(BPayload),

Convert JSON payload to binary

  Token = "7518b1c2c7686d3b5dcac8232313d5d0047cf0dc0ed5d753c017ffb64ad25b60",
  BToken = hex:hexstr_to_bin(Token),
  BTokenLength = erlang:byte_size(BToken),

Convert token from hexadecimal string to binary

  SomeID= 1,
  {MSeconds,Seconds,_} = erlang:now(),
  Expiry = MSeconds * 1000000 + Seconds + 3600*1,

Transaction id (can be always 0) and 1 hour  expiration time

  Packet = <<1:8, SomeID:32/big, Expiry:32/big, BTokenLength:16/big, BToken/binary, PayloadLen:16/big, BPayload/binary>>,

Construct the binary packet.

  ssl:send(Socket, Packet),
  ssl:close(Socket).

Send the PDU and close the socket

3. Listening for Errors

In case something went wrong, Apple will send you back single error packet for the first error and closes the socket. You need to read that one error code. The packet that triggered error is identified by the ID you set when sending it.

See table 5-1 at Apple documentation to interpret error codes.

Example error listener

recv(Parent) ->
   receive
       {ssl, Sock, <<Command, Status, SomeID:32/big>>} ->
           error_logger:error_msg("Received", 
                                  [Command, Status, SomeID]),
           ssl:close(Sock),
           Parent ! {error, SomeID}; % notify parent
      {ssl_closed, _Sock} -> ok  %
   end.

And remember to spawn process and set it as the controlling process after creating the socket

  Pid = self(),
  ssl:controlling_process(Sock, spawn(fun() -> recv(Pid) end)),

Note that you need to implement also poller application to read feedback info from Apple Feedback server. This is very similar to the receiver above as it only needs to connect and wait for packets from Apple server until it closes the socket. See Apple documentation for more in depth explanation.

 

Apple Push Notifications with Node.js

When your iPhone app backend  needs to send Apple Push Notifications, it must do this over raw SSL socket using Apple proprietary raw binary interface. Standard Web REST is not supported. This kind of sucks, because if your entire backend is web based you need to break that cleanliness with external HTTP to APN proxy. One option is to use services like Urban Airship, but you can also build the proxy by yourself.

One potential platform for this is hyped Node.js, the rising javascript engine for building ad-hoc web servers. Web is full of examples of building simple HTTP based server or proxy with Node.js, so this post is only the part where we open a secure connection to the Apple server and send push notifications with plain Node.js javascript.

Please note that Apple assumes that you pool and keep sockets open as long as you have notifications to send. So, don’t make naive implementation that makes new socket for each HTTP request. Some simple pooling and reuse is a must for real implementation.

In addition for sending the push notifications, your app also needs to poll the APNS feedback service to find out what devices have uninstalled the app and should not be pushed new notifications. See more details in post Apple Push Notification feedback service.

1. Get Certificates

Apple’s Push notification server authenticates application by SSL certificates. There is no additional authentication handshake after secure connection has been established.

First we need the PEM format certificates that you can get by exporting  them with Apple Keytool. Export also the Apple Worldwide CA certificate. See this excellent blog post (up to step 5)  for details how to acquire the PEM files: http://blog.boxedice.com/2010/06/05/how-to-renew-your-apple-push-notification-push-ssl-certificate/

Now you should have following certificate files.

  • app-cert.pem  (Application cerificate)
  • app-key-noenc.pem  (Application private key)
  • apple-worldwide-certificate-authority.cer  (Apple CA certificate)

2. Open Connection to Push Server

UPDATE:See more complete TLS example here.

Moving on the actual implementation in Node.js. This is quite simple, you just read the various certificate files as string and use them as credentials.

You must also have SSL support built in your Node.js binary.

var fs = require('fs');
var crypto = require('crypto');
var tls = require('tls');

var certPem = fs.readFileSync('app-cert.pem', encoding='ascii');
var keyPem = fs.readFileSync('app-key-noenc.pem', encoding='ascii');
var caCert = fs.readFileSync('apple-worldwide-certificate-authority.cer', encoding='ascii');
var options = { key: keyPem, cert: certPem, ca: [ caCert ] }

function connectAPN( next ) {
    var stream = tls.connect(2195, 'gateway.sandbox.push.apple.com', options, function() {
        // connected
        next( !stream.authorized, stream );
    });
}

3. Write Push Notification

After secure connection is established, you can simply write push notifications to the socket as binary data. Push notification is addressed to a device with 32 byte long push token that must be acquired by your iPhone application and sent to your backend somehow.

Easy format is simple hexadecimal string, so we define first a helper method to convert that hexadecimal string to binary buffer at server side.

function hextobin(hexstr) {
   buf = new Buffer(hexstr.length / 2);
   for(var i = 0; i < hexstr.length/2 ; i++) {
      buf[i] = (parseInt(hexstr[i * 2], 16) << 4) + (parseInt(hexstr[i * 2 + 1], 16));
   }
   return buf;
 }

Then define the data you want to send. The push payload is a serialized JSON string, that has one mandatory property ‘aps’. The JSON may contain additionally application specific custom properties.

var pushnd = { aps: { alert:'This is a test' }};
// Push token from iPhone app. 32 bytes as hexadecimal string
var hextoken = '85ab4a0cf2 ... 238adf';  

Now we can construct the actual push binary PDU (Protocol Data Unit). Note that payload length is encoded UTF-8 string length, not number of characters. This would be also good place to check the maximum payload length (255 bytes).

payload = JSON.stringify(pushnd);
var payloadlen = Buffer.byteLength(payload, 'utf-8');
var tokenlen = 32;
var buffer = new Buffer(1 +  4 + 4 + 2 + tokenlen + 2 + payloadlen);
var i = 0;
buffer[i++] = 1; // command
var msgid = 0xbeefcace; // message identifier, can be left 0
buffer[i++] = msgid >> 24 & 0xFF;
buffer[i++] = msgid >> 16 & 0xFF;
buffer[i++] = msgid >> 8 & 0xFF;
buffer[i++] = msgid > 0xFF;

// expiry in epoch seconds (1 hour)
var seconds = Math.round(new Date().getTime() / 1000) + 1*60*60;
buffer[i++] = seconds >> 24 & 0xFF;
buffer[i++] = seconds >> 16 & 0xFF;
buffer[i++] = seconds >> 8 & 0xFF;
buffer[i++] = seconds > 0xFF;

buffer[i++] = tokenlen >> 8 & 0xFF; // token length
buffer[i++] = tokenlen & 0xFF;
var token = hextobin(hextoken);
token.copy(buffer, i, 0, tokenlen)
i += tokenlen;
buffer[i++] = payloadlen >> 8 & 0xFF; // payload length
buffer[i++] = payloadlen & 0xFF;

var payload = Buffer(payload);
payload.copy(buffer, i, 0, payloadlen);

stream.write(buffer);  // write push notification

And that’s it.

4. Handling Error Messages

Apple does not return anything from the socket unless there was an error.  In that case Apple server sends you single binary error message with reason code (offending message is identified by the message id you set in push message)  and closes connection immediately after that.

To parse error message. Stream encoding is utf-8, so we get buffer instance as data argument.

stream.on('data', function(data) {
   var command = data[0] & 0x0FF;  // always 8
   var status = data[1] & 0x0FF;  // error code
   var msgid = (data[2] << 24) + (data[3] << 16) + (data[4] << 8 ) + (data[5]);
   console.log(command+':'+status+':'+msgid);
 }

This implementation assumes that all data (6 bytes) is received on single event. In theory Node.js might return data in smaller pieces.

5. Reading Apple Feedback notifications

Apple requires that you read feedback notifications daily, so you know what push tokens have expired or app was uninstalled. See this blog post Polling Apple Push Notification feedback service with Node.js for details.