What's new in Socket.IO 1.0

I first stumbled upon Socket.IO when I was writing my book. The chapter was devoted to making a multiplayer version of a standalone HTML5 game. I built up a solid plan: describing transports, implementing ajax-longpoll, some notes about WebSockets (it was 2012, so WebSockets were not yet production-ready especially for mobile devices), and finally writing some logic on top of it. Then I saw Socket.IO and... why would I spend fifty pages writing about transports if I could spend five writing about Socket.IO instead?

This tool was simply great. The API was clean and intuitive and coding client-server calls felt natural. Of course it had its own problems. I remember spending an evening to debug it with my Samsung Galaxy v1 that was randomly breaking connections, but it worked at the end.

Then I forgot about it for some time. Couple of months ago I was conducting a "Node.js Bootcamp" training and Socket.IO was a part of my program. During the presentation one of my students told: "Socket.IO is dead, Guillermo is not updating it, just look at github page". And indeed, there was not to much activity in the github project.

Luckily, this was in no way the end of life for Socket.IO but rather the preparation for a major rebirth. Socket.IO 1.0 has landed just few weeks ago to show everyone that "the event emitter of the web" is alive and well.

This post is about the changes in Socket.IO 1.0 compared to the older versions. It covers both the changes in the architecture, npm module ecosystem as well as changes in API.

What's Socket.IO

Socket.IO is a JavaScript library that implements a real time communication between a browser and a server. It abstracts-out the details of underlying transport and provides some higher level abstractions like broadcasts, namespaces and rooms. Socket.IO is both a client and server library. The server by default is Node.js, however other third party ports are available. For example, you can run a Socket.IO server in Java and connect to it from the native ObjectiveC client.

The basic code for both client and a server is very straightforward and based on the idea of emitted events. You emit events on one side of the wire and get them on the other. As simple as that.

Client:

var socket = io();
  socket.emit('greeting', 'Hello, my name is Billy');
  socket.on('message', function(msg) {
  console.log(msg);
});

Server:

io.on('connection', function(socket){
  socket.on('greeting', function(text) {
      console.log(text);
      socket.emit('message', 'glad to see you');
  });
});

This code (omitting some trivial setup) is enough to start communication between a client and a server. The client initiates a connection and sends a greeting to the server. Server gets the greeting and replies with "glad to see you".

Note. If you did not write any Socket.IO apps before, a great place to start is the official tutorial that shows how to create a simple chat application - http://socket.io/get-started/chat/

New Modules

Before 1.0 Socket.IO was distributed as solid npm package that implemented every bit of functionality inside.

In Socket.IO 1.0 the code broke up into several small reusable modules less dependent on each other. The most notable shift is the extraction of low-level transport details into a separate project called Engine.IO. Now Socket.IO's job is to provide only higher-level facilities: multiplexing, reconnection, rooms, broadcasting, etc. All the heavy-lifting related to transports is done by Engine.IO.

This change is purely internal. If you are writing your code against Socket.IO you will not see the difference, as Engine.IO is not exposed "as is" to the client code. It is and important and very well-thought architectural change as it makes the overall project more maintainable. Besides, now there's a clear borders of components' responsibilities: Socket.IO is doing logic, Engine.IO is doing communication.

Socket.IO modules Ecosystem

Socket.IO and Engine.IO both are broken into core (socket.io and enigne.io), clients (socket.io-client and engine.io-client) and parsers (socket.io-parser and engine.io-parser).

The official part of the ecosystem looks like in the diagram below. The bold lines show actual NPM dependencies while thin black lines show logical relations. I omitted the repeating parts in the names of the projects, so -client in socket.io is socket.io-client and in engine.io it is engine.io-client.

This section describes modules behind each of ecosystems. We start from smaller and more independent modules and then move to higher-level abstractions.

  • engine.io - a transport engine. Abstracts-out the details of exchanging messages between a client and a server. It defines Socket - a class that hides the details of transports used by a client at a given time. This module low-level. For example there's no notion of types of events or rooms: only strings and binary arrays that travel between a client and a server. High level details are left for Socket.IO. You can use Engine.IO as a standalone server however in most cases it is better to use Socket.IO. This module uses engine.io-parser to encode and decode messages.

  • engine.io-client - a client implementation for engine.io that can run both in browser and Node.js. Uses engine.io-parser to encode/decode messages.

  • engine.io-parser - an implementation of engine.io protocol. Includes both encode and parse functions. Engine.IO uses a simple protocol to exchange binary or utf-8 messages. If there's no binary support on the platform, message is passed as Base64 string. The only meta-data sent along with the message is message type: ping, pong, data, etc. Parser can also work with batches: encode or decode multiple messages as one packet.

  • socket.io - that's what this post is all about. The high-level event transmitter that implements re-connects, rooms, broadcasts, multiplexing and other advanced features. It relies on engine.io for low-level transport, socket.io-parser for protocol handling, socket.io-client for client-side code and socket.io-adapter for tracking rooms and namespaces.

  • socket.io-adapter - having somewhat misleading name, an Adaptor is a registry of users, rooms and namespaces. It has only 4 methods:

    • add(): for adding a socket to a room
    • del(): to delete a socket from a room
    • delAll() - to delete a socket from all rooms
    • broadcast() - to send a message to all sockets in a room this class is a base for extensions. socket.io-redis builds a redis-based Adaptor that allows to distribute Socket.IO application between multiple nodes.
  • socket.io-client - a client for Socket.IO. You can use it from both browser and Node.js. Uses socket.io-parser to encode and decode messages

  • socket.io-parser - a parser and encoder of Socket.IO protocol, a reference implementation of socket.io-protocol. It is not an extension of engine.io protocol. It runs on top of engine.io just like REST is a protocol on top of HTTP and not an extension of HTTP. Socket.IO protocol describes the transformation from/to Engine.IO transportable entity plus Socket.IO specific fields like packet type or namespace for multiplexing.

  • socket.io-protocol - a description of Socket.IO protocol (not the implementation, this repository has only a readme.MD in it). This protocol is higher-level than the protocol of Engine.IO. For example, it supports sending different types of data: strings, JSON objects or binary data. Besides it supports matching requests and responses and marking messages for rooms.

  • socket.io-redis - a Redis-based implementation of an Adapter. With the help of this module you can set up a cluster of Socket.IO instances that will use Redis for communication between nodes.

  • socket.io-emitter - a Socket.IO-independent Node.js client that can send messages to redis-powered cluster

New Transport strategy

Before 1.0 Socket.IO was acting optimistic and first tried to establish the WebSocket connection. If that failed, Socket.IO downgraded to XHR. This approach could cause long delays if the environment was blocking WebSockets. Sometimes a client had to time out to know that WebSocket is not available.

Engine.IO implements a pessimistic but more user-friendly strategy. It starts from XHR longpoll and upgrades to WebSockets afterwards. Since XHR is available everywhere and less likely to cause problems, you get an immediate connection to the server and start to send and receive messages while engine.io is upgrading your connection. Over the last years this strategy has proven to bring better user experience most of the times.

New API

Socket.IO introduced several big changes to the architecture and API: middleware, finer-grained logging and client API changes.

Let's start with middleware as it is the most notable change. Just like in Express.js you can now register a function that is executed once the socket is created:

io.use(function(socket, next){
  count++;
  next();
});

This function can also perform more sophisticated logic like handling authorization, auto-joining some rooms or adding extra listeners for statistics. If you have some code in your Socket.IO application that does not implement core logic but rather handles an auxiliary aspect (authorization, logging, statistics, filters), it is a good candidate for becoming a middleware.

The other change is logging strategy. Socket.IO used to be quite verbose and it was lacking low-level control of individual components console output. Starting from version 1.0 Socket.IO is debug (https://www.npmjs.org/package/debug). Now it is a pleasure to configure logging. Just set a DEBUG environment variable and you get an output only from the components that you need. For example, this is the way to print only what a Socket object is having to say:

set DEBUG=socket.io:socket

There are more changes related to API, but since they are well listed in the official documentation, I will just leave a reference to the migration page: http://socket.io/docs/migrating-from-0-9/.

Summary

Summarizing the changes, Socket.IO 1.0 became cleaner, smarter and easier to extend than its 0.9x predecessors.

Links

Below are some links to github repositories that I referred in this post.

Comments

Please add your comments on github, or twitter @juriy.

Don't forget to follow, this will surely encourage more great posts!

Switching to static

I've got rid of Wordpress. During 8 years of blogging I was fighting with it. Everybody else around was using WP I thought that the problem was in me and all that pain is an essential part of tech writing. Wordpress was getting bigger and bigger and now it looks like an overly-complex engine for a small thing like a blog with several dozens of posts. It is still great for something more complex, but for serving several dozens of posts it is too much. For me personally the most useful feature of WP was commenting system but high volumes of spam drive standard Wordpress comments almost useless.

Docpad logo

The setup that I'm using right now is based on a great tool called Docpad (http://docpad.org). This generator of static pages takes a template and a bunch of posts and creates a static site that can be served by nginx, github pages, Heroku or any other service capable of hosting static content. Templates allow to insert just enough logic to do the tasks that I had so far. CSS and HTML is all hand-written by myself. Finally, I feel like I have control over my content.

What's best about this approach is that I can keep my posts in a in a writer-friendly markdown format and edit them any time. I hated to propagate changes from my drafts to online text editor that stores a post in a database. I hated to backup database each time I needed to save my posts. I hated to apply endless security patches. Now I have no database, my editor is Sublime Text and my site is as secure as nginx is.

To make setup even neater I’ve created a github repository for posts and added a webhook that calls a node.js webserver each time someone pushes new content. Node.js server in turn does git pull and re-generates the content of a blog. That’s quite close to a publishing system that I’ve dreamt of. The only thing that I’m missing is a way to edit and push new posts from my iPad, but I’ll figure out how to do that too.

Docpad is not the only tool in a family. In fact, there are many such tools. There's even a website that compares the most popular frameworks in each language and platform: http://staticgen.com/ so if you like the idea I recommend to take a brief look before choosing a tool that is right for you. In any case, if decide to change a tool later it should be relatively straightforward since all your posts and templates are independent from the generators.

Trip to Hong Kong

After almost half of a year waiting, doing paperwork, packing, unpacking and re-packing bags I'm finally here in Hong Kong. EPAM - the company where I work for almost three years, acquired JoinTech, a software vendor based in Shenzhen. Six adventurers counting me, my wife and daughter landed in Hong Kong on Sunday, May 11 ready for something completely different from what we've experienced before.

I have been to many places, some of them quite exotic lice Cuba but never in Asia. Probably that's why it felt so astonishingly unexpected, irrational, bright, inconvenient and beautiful.

May is a middle of rain season in this part of the world. The air is warm and hot, first several minutes it's even slightly hard to breeze. Rains are heavy and unexpected. They start suddenly, in thirty seconds torrents of water are flooding across the stree and in five more minutes it is all gone. Or it may rain for couple of hours. You never know.

Hong Kong is a city of contrasts and I've learned right in a first hour in a city. We drove through the gorgeous skyscrapers looking like they came straight from the pages of early Hibson's cyberpunk novels and right into west Kowloon - a mix of bazar and nineteenth century industrial region. An adhesive smell of burned oil, spices and sweat had nothing to do with glass'n'concrete skyscrapers of Hong Kong island. Yes, we were lucky enough to rent a hotel right in the middle of that district.

Hotel is a separate story. Fourteen square meters is quite enough for three people, right. About two square meters left free in the middle of the rest – beds, table, and a tiny cabinet fill everything else. An important advice for those who travel here - check feedbacks before you rent something. Luckily, we had only 5 nights to spend there and our next apartments in Shenzhen were the most luxurious place that I lived in.

We spent our first week in Honk Kong waiting for our Chinese visas. It felt like a month because of many new impressions that we experienced these days. Sometimes we felt lost in this fusion of Western and Asian ways of life but we always found our ways thanks to a wonderful and friendly attitude of people around us. They are ready to help whenever you need it.