Socket binary data stream parsing in Erlang

Erlang code has two different ways of reading from the socket, active and passive. In passive mode, your code calls recv to the socket to receive bytes. In active mode you install controlling process to the socket and receive data as Erlang messages.

Binary data parsing is more complicated on the latter, as application code doesn’t have any control over size of the packets (or flow control for that matter) that it receives. It could be 1 byte, few kilobytes or whatever. Naive pattern matching fails if you don’t get exactly the amount of bytes you want.

Lets assume simple binary protocol, where you need to read packets that are varying in length and formatted as following. Timestamp is defined in first 4 bytes, next 2 bytes define length of payload followed by the payload itself.

|    Timestamp   |  Len  |    Payload      |
|  0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | ... |

Incidentally, this is the data format used by Apple Push notification feedback service.

First lets define a function that accepts binary Data and process Pid where it sends parsed result. This tail recursive function simply matches as many packets from data as possible, and returns with remaining unparsed data.

match_data(Data, Parent) ->
   case Data of
      <<Timestamp:32/big, Size:16/big, PushToken:Size/binary-unit:8, Rest/binary>> ->
         Parent ! {Timestamp, PushToken}, % notify parent
         %% parse rest of the data
        match_data(Rest, Parent);
      Rest ->
         %% no match

Then function that actually receives the data from the socket. It receives arbitrary pieces of data, concatenates it to existing unparsed data and calls the match_data handler to make actual packet matching. Then it loops again with unparsed portion of data.

loop(Bin, Parent) ->
      {_, _Sock, Data} ->
         loop(match_data(erlang:list_to_binary([Bin, Data]), Parent), Parent);
      {ssl_closed, _Sock} -> ok;
      {_event, _Event} ->
         error_logger:error_msg("Unexpected", [_event, _Event])

Install the loop as controlling process, with initially empty “seed” data.

Pid = self(),
ssl:setopts(Sock, [{active, true}, {mode, binary}]),
ssl:controlling_process(Sock, spawn(fun() -> loop(<<>>, Pid) end)).

This way data is parsed correctly, no matter what size of chunks data is returned from the socket.

Apple Push Notifications with Erlang

Continuing from the Node.js based example I wrote earlier, here is example how to do the same with Erlang. You can check more details from the previous post, but as reminder the Apple Push Notification interface is simple binary based protocol that you use over SSL authenticated socket.

1. Prerequisites

I assume you have erlang installed, the version I’m using here is Erlang R13B03 (erts-5.7.4).

Check instructions here at Node.js based example how to get the push certificates as .pem files.

Install mochiweb package in your erlang environment.

Check that you’ve all set.

$ ERL_LIBS=. erl
Erlang R13B03 (erts-5.7.4)  [64-bit] [rq:1] [async-threads:0] [hipe] [kernel-poll:false]

Eshell V5.7.4  (abort with ^G)
2> mochijson:encode("cat").
3> application:start(ssl).

Later releases of Erlang may require you to start ‘crypto’ and ‘public_key’ applications before starting ssl.

2. Sending Push Notification

First code to convert hexadecimal strings to binary format. This is mainly for readability for the example. I don’t remember where I snacked that code, but it seems to be found from several sites around the Intertubes.


bin_to_hexstr(Bin) ->
   lists:flatten([io_lib:format("~2.16.0B", [X]) ||
                  X <- binary_to_list(Bin)]).

hexstr_to_bin(S) ->
   hexstr_to_bin(S, []).
hexstr_to_bin([], Acc) ->
hexstr_to_bin([X,Y|T], Acc) ->
   {ok, [V], []} = io_lib:fread("~16u", [X,Y]),
   hexstr_to_bin(T, [V | Acc]).

Then the code to actually connect to APN and send the PDU


sendpush() ->
  Address = "",
  Port = 2195,
  Cert = "cert.pem",
  Key = "key-noenc.pem",  

  %Options = [{cacertfile, CaCert}, {certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Options = [{certfile, Cert}, {keyfile, Key}, {mode, binary}],
  Timeout = 1000,
  {ok, Socket} = ssl:connect(Address, Port, Options, Timeout),

Open SSL socket to the APN server with application certificate and private key.

  Payload = mochijson:encode({struct, [{"aps", {struct, [{"alert", "This is Message"}]}}]}),
  BPayload = erlang:list_to_binary(Payload),
  PayloadLen = erlang:byte_size(BPayload),

Convert JSON payload to binary

  Token = "7518b1c2c7686d3b5dcac8232313d5d0047cf0dc0ed5d753c017ffb64ad25b60",
  BToken = hex:hexstr_to_bin(Token),
  BTokenLength = erlang:byte_size(BToken),

Convert token from hexadecimal string to binary

  SomeID= 1,
  {MSeconds,Seconds,_} = erlang:now(),
  Expiry = MSeconds * 1000000 + Seconds + 3600*1,

Transaction id (can be always 0) and 1 hour  expiration time

  Packet = <<1:8, SomeID:32/big, Expiry:32/big, BTokenLength:16/big, BToken/binary, PayloadLen:16/big, BPayload/binary>>,

Construct the binary packet.

  ssl:send(Socket, Packet),

Send the PDU and close the socket

3. Listening for Errors

In case something went wrong, Apple will send you back single error packet for the first error and closes the socket. You need to read that one error code. The packet that triggered error is identified by the ID you set when sending it.

See table 5-1 at Apple documentation to interpret error codes.

Example error listener

recv(Parent) ->
       {ssl, Sock, <<Command, Status, SomeID:32/big>>} ->
                                  [Command, Status, SomeID]),
           Parent ! {error, SomeID}; % notify parent
      {ssl_closed, _Sock} -> ok  %

And remember to spawn process and set it as the controlling process after creating the socket

  Pid = self(),
  ssl:controlling_process(Sock, spawn(fun() -> recv(Pid) end)),

Note that you need to implement also poller application to read feedback info from Apple Feedback server. This is very similar to the receiver above as it only needs to connect and wait for packets from Apple server until it closes the socket. See Apple documentation for more in depth explanation.