How to build and test your Rest API with Node.js, Express and Mocha

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

If you need to build an API, you want it done quickly and simply and you love JavaScript, then this article is for you!
I’m going to show a simple, quick and effective way to build and unit test your API and your routes, using Node.js, Express and a few amazing test libraries: Mocha, SuperTest and should.js that integrate very well together in a natural fashion.
The API style is Rest, that is it leverages URLs and HTTP verbs, but I won’t go much into details in this tutorial.
If you ever used technologies such as WCF, you will be amazed at how much quicker it is to build your API with Node, there’s virtually no machinery to put in place, no service references to update, no WSDL… it’s pretty much just logic to implement!
Of course when you want to design your API around a Rest concept, it is very important to think about the access paths to your resources, they NEED to make sense and to be well structured, otherwise you just end up walking down the good old RPC path.
For a nice intro to this concept, see http://www.slideshare.net/domenicdenicola/creating-truly-res-tful-apis
By the way, in this tutorial I won’t cover advanced topics such as securing the API through HMAC or OAuth solutions.

From words to code:

A nice and free dev environment where to start coding is cloud9 (https://c9.io), as  it offers everything you need and maybe more: a text editor with syntax highlighting, a Node.js environment, a debugger (!), a shell, cloud storage and it connects to your GitHub repository… because you have a GitHub repository, don’t you? We don’t need more than that to write our first API, so let’s start!

Ok we want to write a HTTP API, so we obviously want our API to be accessible via some sort of webserver, but as you may already know, Node.js doesn’t come with its own webserver.
Instead it comes with some facilities built-in; on top of these facilities lies Express.js, the web server of choice for this tutorial. Express.js (http://expressjs.com/) describes itself as a web application framework for node, and I think you should use Express too, as it is a solid and well tested piece of software, easy to setup and test.

After reading some tutorials around, I came up with a structure of 4 short files  that setup and run the API:

1. config.js exports an object that contains all configuration keys and values, pretty much like web.config in an Asp.Net web app. Following this line we can think about having a config-debug.js  and a config-release.js. We’ll see later why.

2. routes.js exports a setup function that takes care of declaring the routes and the handlers that will manage each request.

3. server.js where we configure Express and we export a start function that takes care of setting up the routes and starting the web server

4. index.js is the entry point of our API. It uses pretty much everything we have defined so far to start the logger (my logger of choice is winston), to connect to a database, if present, and to finally start the Express server.

config.js

	module.exports = {
	"db": {
	"mongodb": "mongodb://username:password@dsXXXXX.mongolab.com:45077/databasename"
	},
	"logger": {
	"api": "logs/api.log",
	"exception": "logs/exceptions.log"
	}
	};

view raw
config.js
hosted with ❤ by GitHub

routes.js

	function setup(app, handlers) {
	app.post('/api/profiles', handlers.account.createAccount);
	app.get('/api/profiles/:username', handlers.account.getAccount);
	app.put('/api/profiles/:username', handlers.account.updateAccount);
	app.del('/api/profiles/:username', handlers.account.deleteAccount);
	app.post('/api/lists', handlers.list.createShoppingList);
	app.post('/api/lists/:id', handlers.list.createShoppingList);
	app.put('/api/lists/:id', handlers.list.updateShoppingList);
	app.get('/api/lists/:userId', handlers.list.getShoppingLists);
	app.del('/api/lists/:id', handlers.list.deleteShoppingList);
	}
	exports.setup = setup;

view raw
routes.js
hosted with ❤ by GitHub

As you can see, setup expects two parameters: app is the Express app and handlers is an object that maps to a set of functions that handles user requests. Each of these functions accepts as parameters the request and response objects: e.g.

function handleCreateAccountRequest(req, res) { … }

server.js

	// *******************************************************
	// expressjs template
	//
	// assumes: npm install express
	// defaults to jade engine, install others as needed
	//
	// assumes these subfolders:
	// public/
	// public/javascripts/
	// public/stylesheets/
	// views/
	//
	var express = require('express');
	var app = express();
	var AccountHandler = require('./handlers/AccountHandler');
	var ShoppingListHandler = require('./handlers/ShoppingListHandler');
	var routes = require('./Routes');
	var fs = require('fs');
	var expressLogFile = fs.createWriteStream('./logs/express.log', {flags: 'a'});
	// Configuration
	app.configure(function(){
	app.use(express.logger({stream: expressLogFile}));
	app.use(express.bodyParser());
	app.use(express.methodOverride());
	app.use(app.router);
	app.use(express.static(__dirname + '/public'));
	});
	app.configure('development', function(){
	app.use(express.errorHandler({ dumpExceptions: true, showStack: true }));
	});
	app.configure('production', function(){
	app.use(express.errorHandler());
	});
	var handlers = {
	account: new AccountHandler(),
	list: new ShoppingListHandler()
	};
	function start() {
	routes.setup(app, handlers);
	var port = process.env.PORT || 3000;
	app.listen(port);
	console.log("Express server listening on port %d in %s mode", port, app.settings.env);
	}
	// *******************************************************
	exports.start = start;
	exports.app = app;

view raw
server.js
hosted with ❤ by GitHub

The first part is just some standard Express setup + Express logging.
Now we can see clearly what the parameters for routes.setup are: app is just the Express instance and handlers contains two objects that point to the different handlers that will be able to handle API requests.
Finally, after declaring and exporting the start function, this module exports the Express instance, which will be used in the tests.

index.js

	var config = require('./Config-debug');
	var winston = require('winston');
	var mongoose = require('mongoose');
	var server = require('./Server');
	// We will log normal api operations into api.log
	console.log("starting logger…");
	winston.add(winston.transports.File, {
	filename: config.logger.api
	});
	// We will log all uncaught exceptions into exceptions.log
	winston.handleExceptions(new winston.transports.File({
	filename: config.logger.exception
	}));
	console.log("logger started. Connecting to MongoDB…");
	mongoose.connect(config.db.mongodb);
	console.log("Successfully connected to MongoDB. Starting web server…");
	server.start();
	console.log("Successfully started web server. Waiting for incoming connections…");

view raw
index.js
hosted with ❤ by GitHub

As already said, index.js is the entry point, that means our API will be executed by running the command

$ node index.js

We can easily configure the API logger using winston and the configuration file, we connect in this case to MongoDB using mongoose, a fantastic tool, and then we start the server using the freshly exported function.
That’s it, you don’t need anything else to setup your API along with a solid logger and a database connection.

Ok I have an API but… how do I test it?

The test framework of choice for this tutorial is Mocha (http://visionmedia.github.io/mocha/) along with Supertest for HTTP assertions (https://github.com/visionmedia/supertest) and Should.js for BDD style tests (https://github.com/visionmedia/should.js). Mocha is a great choice because it makes async test natural and fluent.

The purpose here is to unit test our routes  (integration tests) and to build the tests so that we will be able to read our test and the results as if they were plain English sentences!
Let’s create a folder named “test” where we will place our test files and our mocha.opts configuration file where we can specify the output style for our tests.
Finally we can create our unit tests for the routes in a file called “test/routes.js”.

The structure of a test with Mocha is simple and verbose:

	var should = require('should');
	var assert = require('assert');
	var request = require('supertest');
	var mongoose = require('mongoose');
	var winston = require('winston');
	var config = require('./config-debug');
	describe('Routing', function() {
	var url = 'http://someurl.com';
	// within before() you can run all the operations that are needed to setup your tests. In this case
	// I want to create a connection with the database, and when I'm done, I call done().
	before(function(done) {
	// In our tests we use the test db
	mongoose.connect(config.db.mongodb);
	done();
	});
	// use describe to give a title to your test suite, in this case the tile is "Account"
	// and then specify a function in which we are going to declare all the tests
	// we want to run. Each test starts with the function it() and as a first argument
	// we have to provide a meaningful title for it, whereas as the second argument we
	// specify a function that takes a single parameter, "done", that we will use
	// to specify when our test is completed, and that's what makes easy
	// to perform async test!
	describe('Account', function() {
	it('should return error trying to save duplicate username', function(done) {
	var profile = {
	username: 'vgheri',
	password: 'test',
	firstName: 'Valerio',
	lastName: 'Gheri'
	};
	// once we have specified the info we want to send to the server via POST verb,
	// we need to actually perform the action on the resource, in this case we want to
	// POST on /api/profiles and we want to send some info
	// We do this using the request object, requiring supertest!
	request(url)
	.post('/api/profiles')
	.send(profile)
	// end handles the response
	.end(function(err, res) {
	if (err) {
	throw err;
	}
	// this is should.js syntax, very clear
	res.should.have.status(400);
	done();
	});
	});
	it('should correctly update an existing account', function(done){
	var body = {
	firstName: 'JP',
	lastName: 'Berd'
	};
	request(url)
	.put('/api/profiles/vgheri')
	.send(body)
	.expect('Content-Type', /json/)
	.expect(200) //Status code
	.end(function(err,res) {
	if (err) {
	throw err;
	}
	// Should.js fluent syntax applied
	res.body.should.have.property('_id');
	res.body.firstName.should.equal('JP');
	res.body.lastName.should.equal('Berd');
	res.body.creationDate.should.not.equal(null);
	done();
	});
	});
	});
	});

view raw
test.js
hosted with ❤ by GitHub

So if you try and read the test, it would come out something like:
Describe Routing, describe Account, it should return error trying to save duplicate username, it should correctly update an existing account, and so on.
To add more test, simply add more describe or it functions, and that’s it!

To execute the tests, start your api with

$ node index.js

and then in another shell run

$ mocha

as by default mocha will run everything in /test off of your main project.

If you want to run your test by typing

$ npm test

then all you have to do is to create a makefile and can even be as follows:

	test:
	@./node_modules/.bin/mocha
	.PHONY: test

view raw
Makefile
hosted with ❤ by GitHub

then add the following lines to your package.json file:

“scripts”: {
“test”: “make test”
}

and that’s the result (note that in the screenshot I have more tests as it’s taken from an actual project of mine)

Mocha test output with BDD style

That’s it, pretty easy, isn’t it?