Node.js + Express + Redis = Web Service

The goal is to provide real-time information from Redis Cache via Node.js to web clients. The data include temperature, ozone levels and humidity, collected from devices in various regions.
A sample record:
key: be65-429a-8706a:region2
value: [{“id”:”be65-429a-8706a:region2″,”x”:175.1319,”y”:174.3768,”ts”:”Thu Oct 15 2015 14:03:33 GMT+0000 (UTC)”,”temp”:”32.7″,”humidity”:”87.2″,”o3ppm”:”0.420″}]

For this demonstration, I used openSUSE 13.2 x86-64 on a single core VM with 1.75GB of RAM and installed Node.js v4.2 via zypper. I also installed redis version 3.0.4 for local testing, but the demonstration allows for connections to a remote Redis Cache. The Redis Cache behind this demonstration contains data about geographic regions and the devices reporting in those regions. The data is defined in the following structures:

• one sorted set that is a list of regions
• key-value pairs of JSON strings that contain information about devices within a region

The key for each key-value pair is appended with the string “:regionNN” where NN is the region id stored in the sorted set. This allows for searching for devices in a region. The Node.js code serves three functions:

• number of records
• list of regions
• list of devices in a region

You can “git” a copy of the code at https://github.com/markfennell/redisNode

Getting Started

Start by creating a working directory

mkdir node-redis

Change into the working directory

cd node-redis

Use npm (node package manager) to install the client for redis and the express framework. For this demonstration, we are not using the express framework to its full potential, but mainly to sort out the requests and handle responses.

npm install redis
npm install express

Next, create a file named server.js and let’s create some code. We need to identify the  requirements with a few variables.

var redis = require("redis");     // need redis connection
var url = require("url");         // need to parse URL
var express = require('express'); // makes things easier

Then, we initiate the express framework.

var app = express(); // init express

Next, we declare a few variables to make switching the Redis connection easier.

var redis_srvr = "redis-test.redis.cache.windows.net"; // redis server url
var redis_key = "redis_auth_key" // redis auth key

Once we have the connection information for Redis, we can create a connection.

var client = redis.createClient(6379, redis_srvr, {"auth_pass": redis_key});

A little error checking on the connection never hurts.

client.on("error", function (e,r){ console.log("redis error: "+e);});

Next, we need to enable cors (cross origin resource sharing) so that non-local clients can consume the service data. If you require security, you can limit the client origins with the Access-Control-Allow-Origin parameter. In this demonstration, express injects the cors header into the response. This code was borrowed with light modification from the enablecors website: http://enable-cors.org/server_expressjs.html

app.use(function(req, res, next) {
 res.header("Access-Control-Allow-Origin", "*");
 res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
 next();
 });

Code to Return Data

Now we get into the code that actually writes the response back to the clients. For the purposes of this demonstration, every request is logged to the console. This is useful for debugging purposes but should be commented out for production. A boolean variable could be declared and if it is true, then use the console.log functions.

We start with a simple “Hello” response to the root document. So, if we point a web browser to http://localhost:3000/, the server will respond with “Hello.” This is useful to ensure that the node server is running. The port 3000 is declared later in the code, but is typically a port that can accept incoming requests without requiring the elevated privileges of sudo or root.

app.get('/', function(req, res) {
 console.log("got request");
 res.send('Hello\n');
 });

One of our key functions is to identify the number of records in the Redis Cache. Since the dbsize command in Redis only returns a number, we added formatting that would allow clients to read the size as a JSON string.

app.get('/dbsize', function(req, res) {
 console.log("got request for DBSIZE");
 client.dbsize(function (err, reply) { res.send("[{\"dbsize\":"+reply+"}]"); 
 console.log(reply);});
 });

To retrieve a list of available regions, client will call the http://localhost:3000/regions url. The server response is a CSV list of regions. The client can then create a drop down list of region values for a user to select from.

app.get('/regions', function(req, res) {
 console.log("got request for list of regions");
 client.zrange(["regions","-9999","9999"], function (err, reply) 
 { res.send(reply.sort()); });
 });

Once a region is selected, the client can call the url http://localhost:3000/regionID/NN where NN is the number selected from the list. Using the Redis keys command and the url parameter, we build a search string and return a JSON string of results. Alternatively, we indicate when no data were returned.

app.get('/regionID/:id', function(req, res) {
    regionID = req.params.id;
    console.log("got request for devices on region: "+regionID);
    client.keys("*:region"+regionID, function(err, reply) 
    { client.mget(reply, function(err2, reply2){
        if(reply.length>=1) {
        res.send("["+reply2.toString()+"]");
        } else {
        res.send("No Data");
        }
    });
    });
});

That concludes the primary functionality of the Node.js program. The only loose ends to wrap up are to set the listening port and a console message to indicate the program started successfully.

app.listen(3000); console.log('Listening on port 3000...');

Consuming Data

To test the Node code, issue node server.js from the command line in the working directory. From a web browser you should be welcomed with “Hello” if you go to http://hostname:3000/

To view the list of regions, go to http://hostname:3000/regions

To view a list of devices in a region, go to http://hostname:3000/regionID/nn where nn is a valid region returned in the regions list.

I used jQuery to consume the web service data. The following JavaScript code updates the divAllDevicesCount div with the dbsize every 10 seconds. I subtract one from the dbsize to account for the sorted set which is included in the dbsize.

var devicesCount = setInterval(function () { 
$.getJSON("http://hostname:3000/dbsize", function (data) { 
$("#divAllDevicesCount").html((data[0].dbsize)-1) }) }, 10000);

A select element can be populated with the list of regions using the following code.

function loadRegions() {
    $('select option').remove();
var jqxhr = $.ajax({   // regions stored as list, not json
    type: 'GET',
    dataType: 'text',
    crossDomain: true,  // cors
    url: "http://hostname:3000/regions/",
    success: function(data, textStatus, jqXHR) {
      $.each(JSON.parse(data), function(index, key) {
        $('#select_Regions')
          .append($("<option></option>")
            .attr("value", key)
            .text("Region: " + key));
 
      });
      var selectList = $('#select_Regions option');
      selectList.sort(function(a, b) { // sort the list
        a = a.value;
        b = b.value;
        return a - b;
      });
      $('#select_Regions').html(selectList);
      $('#select_Regions').prepend("<option value='' disabled>Choose a Region</option>").val('');
    }
  });
}

Conclusion

Using Redis as a fast data storage engine, Node.js as a lean web application service and jQuery as a thin client, we can efficiently serve data from the Internet of Things (IoT).