How to secure your HTTP API endpoints using Facebook as OAuth provider

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

This post is the natural follow-up of my post “How to build and test your Rest API with Node.js, Express and Mocha”.
Here I’ll show you a possible solution to the problem of authenticating your API users via Facebook, accessing some user’s basic info like name, profile pic and email and using the given Facebook access token to secure all your API endpoints.

Solution:

This is quite a common scenario nowadays yet I couldn’t find many useful source of information. So I went ahead and asked a question on StackOverflow (http://stackoverflow.com/questions/15602667/possible-approach-to-secure-a-rest-api-endpoints-using-facebook-oauth). The solution described below is heavily inspired by the answer given by user Tommy Crush, so thanks Tommy!

The logical steps are:

  1. Login via Facebook on your app (either mobile or web) and obtain the Facebook Access Token

  2. Pass the Facebook Access Token to the server (over SSL for the android app, and for the web app just have it redirect to an API endpoint after FB login)

  3. Check the given Facebook Access Token to make sure it is valid by making a graph call to the /me endpoint. Parse the response to get user id and email and cross-reference this with existing users to see if it’s a new or old one.

  4. Now create a random token, store it and associate it to the user id and give the API token back to the requesting app. If you need Facebook for anything other than login, store it too in your DB and associate it with the new api_access_token and your user_id.

  5. For every call hereafter, send the api_access_token to authenticate it. If you need the fb_access_token for getting more info, say retrieveing the list of your user’s friends, you can do so by retrieving it from the DB.

Code:

The code shown below  is taken from my current project ShopWithMe available on GitHub  (https://github.com/vgheri/ShopWithMe) and uses a Node.js + Express.js + MongoDB with mongoose technology stack.

I’ll go into details starting of points 3 and 5. Points 1, 2 and 4 should not be a issue.

Step 3: Verify facebook access token

To verify if the given token is valid we need to make a call to the /me endpoint of the Facebook Graph API located at the URL https://graph.facebook.com/me?access_token=$fb_access_token
The result, if token is valid, is a JSON blob containing the user’s information accessible using this token.
This blob should, at its minimum, look like the following:

{
  "id": "xxxxxx",
  "name": "XXX XXX",
  "first_name": "XXX",
  "last_name": "XXX",
  "link": "https://www.facebook.com/XXX",
  "username": "xxxxxx",
  "gender": "xxx",
  "email": "xxx",
  "timezone": 2,
  "locale": "en_US",
  "verified": true,
  "updated_time": "2013-08-14T09:16:58+0000"
}

Otherwise should we have an error response, that’s the JSON blob we’ll get back:

{
  "error": {
    "message": "XXX",
    "type": "OAuthException",
    "code": XXX,
    "error_subcode": XXX
   }
}

As a hint, if you want to try this on the browser, just go to https://developers.facebook.com/tools/explorer/?method=GET&path=me being logged in with Facebook on your browser. Obtain the Facebook Access Token and then point your browser to the URL https://graph.facebook.com/me?access_token=$fb_access_token , where $fb_access_token is the token you just obtained. You should see a JSON object containing your info if all went well (why shouldn’t it? J)

Point 5:
Point 5 is about securing your API endpoints, making sure that only authenticated users can access the data and authorize the request.
Express.js has a nice and easy way to allow a developer to play with the request pipeline, all that is needed is adding a function in the route declaration.
Let’s see how:
In the file where I defined my routes, I just need to update the routes I want to secure, simply modifying them like this:

BEFORE:
app.get(‘/api/profiles/:userId’, handlers.account.getAccount);

AFTER:
app.get(‘/api/profiles/:userId’, authorisationPolicy, handlers.account.getAccount);

As it should be clear, I just added a function called authorisationPolicy to the route declaration that inside the pipeline is processed ahead of the handlers.account.getAccount function .
Let’s see into detail this authorisationPolicy function:

This function executes its authentication policy and if everything is fine, it calls the Express.js next() function, which takes care of executing the next stage in the pipeline.
If otherwise the request should not be authorized, we can send back an error response.

Conclusion

In conclusion what we have seen is a simple yet effective way to integrate Facebook login into your HTTP API and to secure your endpoints.
The key here is to use the Facebook token to identify your users and then to issue your own API tokens to secure your endpoints.

How to build and test your Rest API with Node.js, Express and Mocha

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

If you need to build an API, you want it done quickly and simply and you love JavaScript, then this article is for you!
I’m going to show a simple, quick and effective way to build and unit test your API and your routes, using Node.js, Express and a few amazing test libraries: Mocha, SuperTest and should.js that integrate very well together in a natural fashion.
The API style is Rest, that is it leverages URLs and HTTP verbs, but I won’t go much into details in this tutorial.
If you ever used technologies such as WCF, you will be amazed at how much quicker it is to build your API with Node, there’s virtually no machinery to put in place, no service references to update, no WSDL… it’s pretty much just logic to implement!
Of course when you want to design your API around a Rest concept, it is very important to think about the access paths to your resources, they NEED to make sense and to be well structured, otherwise you just end up walking down the good old RPC path.
For a nice intro to this concept, see http://www.slideshare.net/domenicdenicola/creating-truly-res-tful-apis
By the way, in this tutorial I won’t cover advanced topics such as securing the API through HMAC or OAuth solutions.

From words to code:

A nice and free dev environment where to start coding is cloud9 (https://c9.io), as  it offers everything you need and maybe more: a text editor with syntax highlighting, a Node.js environment, a debugger (!), a shell, cloud storage and it connects to your GitHub repository… because you have a GitHub repository, don’t you? We don’t need more than that to write our first API, so let’s start!

Ok we want to write a HTTP API, so we obviously want our API to be accessible via some sort of webserver, but as you may already know, Node.js doesn’t come with its own webserver.
Instead it comes with some facilities built-in; on top of these facilities lies Express.js, the web server of choice for this tutorial. Express.js (http://expressjs.com/) describes itself as a web application framework for node, and I think you should use Express too, as it is a solid and well tested piece of software, easy to setup and test.

After reading some tutorials around, I came up with a structure of 4 short files  that setup and run the API:

  1. config.js exports an object that contains all configuration keys and values, pretty much like web.config in an Asp.Net web app. Following this line we can think about having a config-debug.js  and a config-release.js. We’ll see later why.

  2. routes.js exports a setup function that takes care of declaring the routes and the handlers that will manage each request.

  3. server.js where we configure Express and we export a start function that takes care of setting up the routes and starting the web server

  4. index.js is the entry point of our API. It uses pretty much everything we have defined so far to start the logger (my logger of choice is winston), to connect to a database, if present, and to finally start the Express server.

config.js

routes.js


As you can see, setup expects two parameters: app is the Express app and handlers is an object that maps to a set of functions that handles user requests. Each of these functions accepts as parameters the request and response objects: e.g.

function handleCreateAccountRequest(req, res) { … }

server.js

The first part is just some standard Express setup + Express logging.
Now we can see clearly what the parameters for routes.setup are: app is just the Express instance and handlers contains two objects that point to the different handlers that will be able to handle API requests.
Finally, after declaring and exporting the start function, this module exports the Express instance, which will be used in the tests.

index.js

As already said, index.js is the entry point, that means our API will be executed by running the command

$ node index.js

We can easily configure the API logger using winston and the configuration file, we connect in this case to MongoDB using mongoose, a fantastic tool, and then we start the server using the freshly exported function.
That’s it, you don’t need anything else to setup your API along with a solid logger and a database connection.

Ok I have an API but… how do I test it?

The test framework of choice for this tutorial is Mocha (http://visionmedia.github.io/mocha/) along with Supertest for HTTP assertions (https://github.com/visionmedia/supertest) and Should.js for BDD style tests (https://github.com/visionmedia/should.js). Mocha is a great choice because it makes async test natural and fluent.

The purpose here is to unit test our routes  (integration tests) and to build the tests so that we will be able to read our test and the results as if they were plain English sentences!
Let’s create a folder named “test” where we will place our test files and our mocha.opts configuration file where we can specify the output style for our tests.
Finally we can create our unit tests for the routes in a file called “test/routes.js”.

The structure of a test with Mocha is simple and verbose:

So if you try and read the test, it would come out something like:
Describe Routing, describe Account, it should return error trying to save duplicate username, it should correctly update an existing account, and so on.
To add more test, simply add more describe or it functions, and that’s it!

To execute the tests, start your api with

$ node index.js

and then in another shell run

$ mocha

as by default mocha will run everything in /test off of your main project.

If you want to run your test by typing

$ npm test

then all you have to do is to create a makefile and can even be as follows:

then add the following lines to your package.json file:

“scripts”: {
“test”: “make test”
}

and that’s the result (note that in the screenshot I have more tests as it’s taken from an actual project of mine)

Mocha test output with BDD style

Mocha test output with BDD style

That’s it, pretty easy, isn’t it?

PongR part 3: client side

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

In part 1 of “PongR – my first experience with HTML5 multiplayer gaming” , I talked about the spirit of this project, the system architecture, technologies used and the design of the game (remember the server authoritative model?)

In part 2 I talked about the server side code and the server loops

This is part 3 and I will talk about the client side code, the client loops and how we draw basic (no sprites, only basic shapes) images on a canvas.

Code and Demo

As always, the code is fully published on github [https://github.com/vgheri/PongR] and a (boring!) demo is available at http://pongr-1.apphb.com . If you don’t have anyone to play with, just open two tabs and enjoy!

Project structure and technologies used

As a quick recap, this project has been realised using Asp.Net MVC, HTML5 (Canvas), Javascript, C#, SignalR for client/server real-time communication and qUnit for javascript unit testing.

PongR is a multiplayer HTML5 game, and this means that the client itself is doing a good amount of work.
The engine is replicated both on the server and on the client, as we have seen in the first part of this article when I talked about the server authoritative model + client side prediction that force us to add logic onto the client.
In the server this logic is built into a class named “Engine” and later on we will go into more details.
In the client this logic is built into a javascript module named PongR.

Javascript  code structure and unit testing

Most of the Javascript client side code is contained into a folder called Js, whose structure is shown below:



the module PongR is inside the file PongR.js. This module exports a public prototype with two public functions needed to setup an instance of the game and to connect to the server.

pongR.PublicPrototype.createInstance = function (width, height, username) { … }
pongR.PublicPrototype.connect = function () { … }

Furthermore the public prototype exports also a set of functions that have been unit tested using qUnit [www.qunitjs.com].
qUnit is very easy to use and the way I set it up is very neat: you can find all the details in this awesome article http://freshbrewedcode.com/jonathancreamer/2011/12/08/qunit-layout-for-javascript-testing-in-asp-net-mvc3/
Basically what you get is an easy way to share certain pieces of the tests layout in an ASP.net MVC3 application, with the possibility to have all the links to start your different tests grouped into one HTML page, being able to start them with just one click.
The downside of doing things this way is that the functions we want to unit test need to be visible from outside the module. In order not to pollute the public prototype too much, the module exports a UnitTestPrototype object, which will contain the references to the functions that I want to unit test.

var pongR = {
    PublicPrototype: { UnitTestPrototype: {} }
};

All the unit tests are contained into PongR_test.js.
A quick example:

And when runned, this is what you get:

Finally, I’m going to talk about RequestAnimationFrameShim.js later in the article, when I’ll focus on the Canvas element.

PongR

Other than what we already saw, this module defines several View Model, related to objects that needs to be simulated. Below we can see a basic diagram of the objects and their connections:

The main object here is clearly Game, as it contains the whole state of a match.
Settings is another interesting enough object, it holds a few configuration variables that are mostly used for client prediction and interpolation (I will cover this topic later on in the post).
These models are widely used by the client logic: for example the physics update loop directly modifies these models at every simulation step.
And this leads us to discuss about the client side update loops.

Client side update loops


Let’s restart the flow and let’s see more in detail what is happening:
when a client enters its username and posts the form, the system redirects the client to the page where the actual game will be displayed. This page will load all the necessary javascript libraries and modules, as well as PongR, and will then perform this piece of code on the document onready event:

createInstance sets up the basic configuration of the game and creates the variable that will host the SignalR object, pongR.pongRHub, and all related callbacks.
Once the SignalR object has been correctly populated, we can invoke the .connect() function on PongR, that will start the SignalR connection and on success we invoke the joined() function, which will be where the server will process the client.
We need to have something after start() because in the server side handler the round trip state is not yet available.
When 2 players are connected and waiting, the server sends an init message to both clients that is handled by the client by the following callback:

https://gist.github.com/4369339

This code initiliases a new Game, the canvas on which the game will be drawn on, an empty list of server updates that will be received throughout the game and the default delta time set to 0.015 ms that corresponds to 66 fps, a keyboard object, which is a slightly modified version of the THREEx keyboard library that I edited to serve my purposes here, and draws the initial scene (the field, the players and the ball).
After completing initialisation, we can perform a 3 seconds countdown so that the match doesn’t start all of a sudden and the players are not yet ready.
At the end of the countdown the startGame function is invoked.
This function is very important because it starts the two client loops responsible of handling the game inputs and rendering.

function startGame() {
    startPhysicsLoop();
    startUpdateLoop();
}

Client side loops

Client physics update loop

Exactly as the server was running a physics update loop, the client is also running a similar loop.
This loops interacts with and modifies directly the View Models that I described earlier.

function startPhysicsLoop() {
    physicsLoopId = window.setInterval(updatePhysics, 15);
};

This loop runs every 15msec and is responsible for

  • updating at each round the delta time, used to compute the movements of the players
  • updating the position of the client
  • updating the position of the ball
  • checking for possible collisions between the ball and the objects of the game, players and ball. If a collision is detected, than the position and the direction of the ball are updated as well.


https://gist.github.com/4369350

Despite the fact that the source I used to create this project puts the collision-checking code into the client update loop, I moved it inside the physics update loop for simplicity. This is obviously not an ideal solution if you want to play sounds on collisions, for example, given that the sound should be played by the update loop.

Client update loop

This loop, unlike the physics loop, is scheduled using a function recently introduced into modern browsers, RequestAnimationFrame.

function startAnimation() {
    requestAnimationFrameRequestId = window.requestAnimationFrame(startUpdateLoop);
};

You can read in detail about this function here  and more here  .
Basically instead of using setTimeout or setInterval, we tell the browser that we wish to perform an animation and requests that the browser schedule a repaint of the window for the next animation frame. Reasons why RequestAnimationFrame is better than old style setTimeout and setInterval for drawing purposes are clearly stated in the above link, but I think it’s important to quote them here:

The browser can optimize concurrent animations together into a single reflow and repaint cycle, leading to higher fidelity animation. For example, JS-based animations synchronized with CSS transitions or SVG SMIL. Plus, if you’re running the animation loop in a tab that’s not visible, the browser won’t keep it running, which means less CPU, GPU, and memory usage, leading to much longer battery life.

Because this function has been recently introuced, it may happen that some browser still don’t support it and that’s the reason of that RequestAnimationFrameShim.js file that we saw at the beginning. It’s a piece of code found on Paul Irish’s blog aticle mentioned above, so credits go to him.
Let’s see the code:

Initially I check that we are not in a post-goal condition, because after a goal a 3 seconds countdown will be performed and we don’t want to update any of our internal data structures during this time.
If we are not, then we can simulate a new step of the game

At every step the canvas must be redrawn, therefore I can safely clear it.
As this game is fairly simple it won’t impact our performances, otherwise it could have been better to have multiple specialised overlapping canvas, where for example we could use one to draw the background that never changes, and therefore needs not to be cleared & redrawn at each step, and so forth…
Furthermore, I need to process client inputs (if any) and accordingly update a meta-structure that contains a list of commands (“up”/”down”). This meta-structure will then be used by the client physics loop and converted in movements.
Every input processed at each loop is stored in a list of commands (pongR.me.inputs) and assigned a sequence number that will be used when the server will acknowledge already processed inputs. For example, if the server acknowledges input 15, then we can safely remove from the buffer all inputs with sequence number equal to or lower than 15, and reapply all non yet acknowledged inputs (remember the client side prediction model?).
Every input is packed and immediately sent to the server to be processed.

if (!pongR.settings.naive_approach) {
    interpolateClientMovements(pongR.other);
}

function interpolateClientMovements is a bit tricky and you can read the code by yourself for a better understanding (or even better you can check the blog article where I took this from), but basically it is trying to interpolate the different positions of the opponent in time so that at each redraw its movements will result more continuous and natural to the eyes of the player.
Imagine that the opponent is currently at position (50,50) and then finally we receive its new position at (50,100), if we naively assign this new position to the opponent, what we will see on the screen is a big leap from last frame and it’s obviously something we don’t want.
I have to say that my implementation is not working that well at the moment, but the idea is there.
Finally, after having handled all inputs, I can draw the new frame by drawing each object on the screen.

Server authoritative model in practice


Each time the server runs its own update loop, the clients receive an update packet.
Each client is then responsible for processing it and updating its internal structures so that the next time the update loops will run, they will see an updated snapshot of the game as dictated by the server.
The function responsible to handle all of this is the SignalR callback updateGame.
Let’s see it in detail:

As I mentioned in part 1 of this blog article, it’s the server responsibility to simulate game state changing events, and a goal event is one of this kind!
This should clarify the meaning of the first lines of code: the client only knows that a goal happened because the score changed!
Then, based on this condition, we need to perform two different tasks:

  1. If one of the two players scored, we need to update the scores (both internally and on the screen), reset the positions of all the objects drawn on the screen, reset the state of the keyboard object (we don’t care of all the keystroke pressed after a goal!) and finally perform a countdown that will start the match once again.
  2. Otherwise
    1. We need to apply client side prediction to update our client position, re-processing all those inputs which have not been acked by the server yet
    2. If we are not using a naive approach we do not directly update the other player position, but we simply update some of its internal properties and then we push the just received update packet into the updates buffer, so that it will be used in the update loop for interpolation
    3. We update the information related to the ball  object


It is interesting to note that the updates buffer is not infinite and we limit it to cover 1 seconds of server updates.

Final considerations and next step

This was my first attempt at creating a game and adding multiplayer didn’t make the task easier for sure!
The graphic is really basic, the game itself is not that entartaining, the network lag is clearly affecting the user experience mostly due to a not optimised networking code and to the impossibility of using WebSockets (AppHarbor doesn’t support them yet), but nevertheless it was very funny and I learned plenty of stuff while working on this project.
I have to say that offering the clients a seamless game over the wire has probably been the hardest part of it, and I’m sure there are things which are not working like they should in my code.
Also I think that Asp.Net MVC doesn’t offer the best in class experience to realise this sort of web app (as expected), whereas I see much more fitting Node.Js because of its event-driven nature: if you think about it, almost everything happening in a game can be seen as an event.
Last but not least, using a single codebase and a single language can greatly help to speed up the process.
I couldn’t cover everything I wanted to in these three posts, they are already monster size, so I encourage you to clone the repository and dig into the code to find everything out (like for example delta-time implementation and time synchronisation).
In the near future I would like to rewrite this project entirely in Javascript using Node.Js and, enhancing the user experience ameliorating the graphics using sprites, making the game funnier (e.g. possibility to use bonuses), adding sounds, upgrading SignalR to 1.0 etc…

References

That’s all about PongR, I hope this can help someone

Valerio

PongR part 2: server side

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

This is part 2 of “PongR – my first experience with HTML5 multiplayer gaming”  (https://thewayofcode.wordpress.com/2012/11/05/pongr-my-first-experience-with-html5-multiplayer-gaming/)

Code and Demo

As always, the code is fully published on github [https://github.com/vgheri/PongR] and a (boring!) demo is available at http://pongr-1.apphb.com . If you don’t have anyone to play with, just open two tabs and enjoy!
Following a screenshot of the game:

PongR

Initial countdown before the game starts

Project structure and technologies used

As a quick recap, this project has been realised using Asp.Net MVC 3, HTML5 (Canvas), javascript, C#, SignalR for client/server real-time communication and qUnit for javascript unit testing.

PongR is a multiplayer HTML5 game, and this means that the client itself is doing a good amount of work.
The engine is replicated both on the server and on the client, as we have seen in the first part of this article when I talked about the server authoritative model + client side prediction that force us to add logic onto the client.
In the server this logic is built into a class named Engine and later on we will go into more details.
In the client this logic is built into a javascript module named PongR.

What we’ll see

While writing this second part of the post, I realised that tackling both the server and the client side in one single post will result in a wall of text.
Therefore in part 2 I will start covering the server side code, that is the authoritative source for our game.
Finally in part 3 I will cover the client side code, and this will allow me to dive deeper into HTML5 canvas, qUnit and so on.
So, let’s start!

Connecting to the server

I think that an intuitive way to dive into the code and understand it, is to to navigate the flow of the system: let’s pretend there are no matches currently played on the server, and that we are a client that lands on PongR page.
When the client enters its username and posts the form, the system redirects the client to the page where the actual game will be displayed, PongR.cshtml. This page will load all the necessary javascript libraries and modules, PongR comprised, and will then perform this piece of code on the document onready event:

The Joined() method is inside the PongRHub class, on the server, and will accomplish the following tasks:

  1. Add user to list of connected users
  2. If waiting list is empty add user to waiting list
  3. Else find an opponent (first in the waiting list) and remove him from the waiting list
  4. Create room and assign both users
  5. Create a group for this room
  6. Setup match (playRoom Id, initial ball direction, player on the left and right etc…)
  7. Notify the group the match can start
  8. Add the game to the list of games that the Engine must simulate


Now, the first time I implemented this method I had a problem in between point 5 & 7: sometimes the clients didn’t receive the signal from the server to setup the match.
So I went back to the project’s wiki and I browsed the source code and I understood where the problem lies: when the server invokes Groups.Add(), it sends a signal to the client and notify it about its appartenance to a certain group.
Therefore sometimes I had race conditions where the second signal arrived before the first one, causing the client to drop the message.
I quickly and roughly fixed the issue adding a Thread.Sleep() call after the Groups.Add calls.
Here is the code for Joined():

I could have found a more elegant solution adding one more call after the joined to split the work, but the focus of this project is not SignalR per se, so I guess it’s ok anyway…
Next I signal clients that the match can start along with the game options, like initial direction of the ball, players info etc., and finally enqueue the game on the server so that its state will start to be processed.

// sends a message only to the clients belonging to the group, invoking the function setupMatch() that is part of the PongR module.
Clients[playRoom.Id].setupMatch(matchOptions);

// Adds the freshly created game to the list of games to be simulated by the server
Engine.AddGame(game);


This is where our story splits in two parts: we now simultaneously have the clients AND the server playing the game.
How does it work? Again the theory behind what follows is taken from http://buildnewgames.com/real-time-multiplayer/ , which in turn comes from Valve’ Source Engine, I just adjusted it to my needs.
Basically both the server and the clients run two loops, and we call them:

  1. The physics update loop
  2. The update loop

Now these two loops do different things and run at different speeds depending if we are on the server or on the client.
Before moving on talking about the loops, it might be useful to describe what kind of messages the client is sending over to the server.
In PongR a client can only move up or down, and a sequence number is associated to each command to uniquely identify it.
So our payload will contain the following info:

  • the id of the game being played
  • the client id that just sent the command
  • the command
    • a sequence number
    • a string identifying the type of movement: “up” or ”down”


The meaning of the sequence identifier is related to the Client-side prediction mechanism and the input buffer that both the server and the client keep. Here are the steps:

  1. The client moves and stores into its buffer the input along with its id
  2. The client sends the input to the server
  3. The server enqueues the input into a buffer
  4. When the server processes the input, it removes it from the buffer
  5. When the server update loop runs (will see it later in more detail), it notifies the client of the latest input that has been processed
  6. The client can remove from its buffer all the input with ID lower than or equal to the one just ack’ed

 

Update Packet

Update Packet with sequence identifier acknowledgement


The payload size could be slimmed down by a lot, but that is again another story (for some ideas, take a look here: [www.buildnewgames.com/optimizing-websockets-bandwidth ] )
So let’s start with the server, the authoritative source of our game.

The server loops

To run the loops at fixed intervals, I used the System.Timers.Timer object, and I set them up like this in the Global.asax:

In the second gist we also see the handlers that will run when the Elapsed event is raised.

The server physics update loop

The server physics update loop runs around 60 times per second, that is every 15 msecs.
Its job is to update, for every tick of the simulation, players and ball position based on the inputs received by the clients and on the law of the physics of the game.
Let’s see what a loop round looks like:

MovePlayer() and UpdateBallPosition() are relatively simple so I won’t show them here.
Shortly, the former modifies the Y coordinate of the player and its direction (up/down), based on the inputs that it has received, while acknowledging every processed input sequence number, and the latter simple modifies the (X,Y) coordinates of the ball based only on its angle (that implictitly gives us the direction). For every step of the simulation, both the ball and the players (if any input arrived) will move by a fixed amount of pixels.
Browsing the code you will notice that both the player and the ball movement are based on something called DELTA_TIME. This is related to the concept of framerate independence and I invite you to read more about that here http://www.scirra.com/tutorials/67/delta-time-and-framerate-independence/page-1.
Briefly, quoting the link above:
“Framerate independent games are games that run at the same speed, no matter the framerate.”

Moving on to

var goal = CheckGoalConditionAndUpdateStatus(game);

we see that it’s the server  who is checking if we have a goal situation, and if it’s the case, we update the score of each player (important, and later we’ll see why), add the timestamp of this event to a collection of goal timestamps and then we proceed to restart the game after a goal, that is really resetting positions of players and ball to initial state and choosing randomly the new initial ball direction.
I’m storing the goal timestamp because on the client we provide visual feedback of a goal, running a countdown after each so that the user has the time to understand what is going on and get ready for the next ball to play. During the countdown the ball won’t obviously move, therefore the server must not continue to simulate as well during this time.
We can see this check implemented in the outer physics loop, that is the method actually invoked by the Time elapsed event handler:

That’s it for the server physics update loop.

The server update loop

The update loop runs every 45 msecs, around 22 times per second, so at a much slower rate then the physics loop.
It’s all about finding a good trade-off between accuracy of the simulation (faster update rates) and network usage and performance (slower update rates).
The update loop job is to send the updated state of the game to the right clients.
To do that, I used an external class, called Notifier, which has one public method that will use SignalR to invoke a javascript function on the client even if we are outside of the context of our Hub. This is a very cool SignalR feature, as it allows for a lot of flexibility. Let’s see how it’s done:

The following line will send the message to the two clients that are part of the SignalR group

context.Clients[game.GameId].updateGame(packet);

Conclusion

That’s it for part 2 of this article. We have covered the connection process and the server update loops. We have seen all the workarounds used to cope with SignalR Groups and to skip simulation when one of the player scored in a match, so that the clients can display a countdown before restarting the match.
Part 3 will be the last and we’ll take a look at the client side code, even though you can start by yourself cloning the repository on github.

PongR – my first experience with HTML5 multiplayer gaming

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

PongR is a Pong like HTML5 multiplayer game.
I started this project to improve my javascript and HTML5 skills, and being a lover of SignalR I thought that it would have been nice to add a touch of multiplayer, even if I had never done such a thing!
Hint: it turns out that adding multiplayer is a HUGE effort, if you want to do it right or even just not completely wrong. 😀
What I still needed to start was a game with an easy model, that required not much graphics skill and that had multiplayer “built-in”… and that’s how I chose Pong!

It is important to note that this is a project I did to learn, and at least for this very first release, the game is really boring(!) and awfully looking 😦
Furthermore this article just describes my experience and my approach: I will point out all the difficulties I came across, all the things I understood, all the points that are still lacking (an awful lot, unfortunately), better technologies that could be used etc… Therefore it’s not a true tutorial about how a HTML5 multiplayer game should be done (I will post some interesting links, though), it’s more about how I put the pieces of the puzzle together and my considerations about HTML5 game development!

This post will be split in two sub-posts, just to avoid having a single, huge, unreadable, wall of text.

  1. The first half is about about the fundamentals: the (simple) math behind the physics model, the architecture of the system, the technologies used and finally the game client/server model, with an insight of how and why I failed hard at my first attempt.
  2. The second half will be about the implementation details: the game simulation engine (both on server side/client side), javascript code structure and unit tests, usage of canvas and more.

The “mathematical” model

First off I started with the mathematical model that drives the game’s physics: player movements, ball movements, collisions with players and the field, goal event.
Players can only move up/down, as per the original Pong game; about the ball, in order to keep things simple, I adopted a basic model: initially it is moving horizontally, right or left .
During the game, the ball can collide either with a player (N.B. if the player is not moving, the ball will only change it’s horizontal direction) either with the field, and the new direction is computed based on the following schema:

For this first version I deliberately chose not to add more complexity, like a variable speed of the ball and players, that in turn can lead to different bouncing angles.

System architecture and technologies used

Because this is a multiplayer game, there will be an important part of server side code and an equally important frontend part (this point is crucial, I will come back to this).
The backend is implemented using Asp.Net MVC and C#, and the realtime communication is managed by SignalR.
The frontend is plain javascript plus some knockout.js to automatically update some parts of the UI.

Two pages: the landing page where the user enters a name and starts the game, and the second is really the game: the pitch and few info on the screen (players’ names and score).
This leads us to talk about the client connection flow:

  1. A client connects to http://pongr-1.appharbor.com, enters a name and submits the form
    1. If there is no match to join, then the client enters into a waiting state
  2. When there are two clients ready to play, the system
    1. Creates a match
    2. Assigns the two clients to the match
    3. Initialise the state of the game
    4. Adds the game it to the list of games that are currently being played
    5. Signals the clients the match can start, along with the initial state of the game
  3. Clients receive the start signal, setup the match and start their partial simulation of the game and draw the scene

Because we can have multiple games going on at the same time, the server needs to know which client belongs to which game (as per point 2b), so that whenever it has to send update about a game, the server will send the message to only the two clients that are playing that match, and not to everyone.
To do this, the server behind the scenes is storing games and players into an in-memory repository, using SignalR Groups feature to be able to send messages only to the right players.
SignalR Groups feature is a powerful, yet strange beast, as we will see in the second part of this post, posing some constraint on the way the system must be designed in order to leverage it.

Game design

Round 1, fight: a completely wrong approach!

As a first approach, because I didn’t know anything about the HTML5 Canvas object, I tried to bypass it entirely using HTML and CSS to draw each frame of the game: the field was a div, the players were divs, even the ball was a div rounded using CSS (border radius…)!! The UI was heavily automatized using Knockout.js.
Each client was simulating its own game, sending updates to the other client (through the SignalR Hub on the server) about its position on a fixed basis.
This means that each client is authoritative regarding its own state: position, score etc… and that the server is only a mere repeater for status updates.
As I quickly understood by playing the game, the first attempt had several fatal issues:

  1. DOM manipulation is really slow and limited when facing typical game requirements (not surprisingly)
  2. After a few seconds of playing, the players were not playing the same game anymore, they were completely out of synch! This is due to the fact that the network is not a perfect channel with no latency, and even with very few delays, the match is rapidly going out of synch.


By googling, I quickly realized that while addressing the first part would turn out to be easy, addressing the second point would be a totally different task.

Round 2 – A new approach: the server authoritative model and its consequences

As a total newbie to game development, I should have probably started following a tutorial from the start, but… it’s good to fail as well, you learn by understanding first hand what works and what does not.
At the end I found a very nice article (which links to other very interesting articles, that I strongly suggest you read, at least for background culture) about multiplayer HTML5 games: http://buildnewgames.com/real-time-multiplayer/
Because the networking architecture of my project is mostly taken from there, and is explained quite well (it’s better to look at the source code though, it will clarify some concepts), I won’t go too much into detail of each step. Once again the goal here is to provide a general idea and to explain my experience.

So what does it mean that the server is authoritative versus the clients in terms of the simulation of the game?
It means that the one and only authority regarding everything that happens in the world is the server: the server is running a simulation of the game, and it updates it based on the inputs that arrive from the clients (key presses, commands etc…).
On a regular basis, the server also notifies the clients of the new game state: all the connected clients  will receive the same update and they will have to update their representation of the game based on what the server computed.
This simple idea will assure that all the connected clients will share almost (we will see later why “almost”) exactly the same representation of the game at any time.
This model is also used to avoid client from cheating: it’s always the server who decides what happens and what doesn’t!

This new approach leads to a new scenario: if before each client was running its own game, now it is the server that is running the game and the clients are left with nothing but to redraw the scene based on current game state.
You can try this approach quite easily but what you get is very poor performance in terms of reactivity: the client presses left, then the client sends a message to the server to notify it about the move, then the server updates the game state and sends the clients a new update, which in turn will redraw the scene, finally moving the player.
This means that after a loooooooooooooooooooooooooooooong wait, the client who moved will see its position change. This is what causes LAG. Not nice.
To avoid these kind of problems, one of the most common techniques is called “Client-side prediction”, and to describe it I can use the words from this very nice article http://www.gabrielgambetta.com/?p=22

“If the game world is deterministic enough (that is, given a game state and a set of inputs, the result is completely predictable),[…], we can assume the inputs we send to the server will be executed successfully. Under this assumption, the client can predict the state of the game world after the inputs are processed, and most of the time this will be correct.
Instead of sending the inputs and waiting for the new game state to start rendering it, we can send the input and start rendering the outcome of that inputs as if they had succeded, while we wait for the server to send the “true” game state – which more often than not, will match the state calculated locally”

This means that now the client is not a dumb spectator anymore, but actually runs some part of the game simulation too!
In the case of PongR, each client is predicting its own position, but because we also have a ball moving and potentially colliding with other objects, then the client needs also to predict every collision with the ball (and this is especially true if you consider sound effects)!
Unfortunately this implies having to duplicate some part of your code, and this can be painful if the languages used in the frontend and in the backend are different, like in this case!
Luckily enough we now have Node.js, which would allow us to have a single code base in a single language, javascript, but this is another story.

There are also other techniques involved, like delta-time to make objects movement independent from the local framerate and client inteporalation to smooth movements of the opponent player. You can find them all detailed in the links I already pointed out.
One final, important, comment: no game state change event is simulated on the client.
Imagine a FPS game, the clients shoots another player and in its own representation of the game, the opponent dies: you see the animation and you get the points… All is good except that as soon as you receive the update from the server, you find out that it’s wrong, you didn’t really kill the opponent as he was 1 meter more on the left that you had known. So… how can you rollback such a situation? Simply you cannot, and that’s why game state change events are only simulated on the server.
In the case of PongR, the only game state change event is when one of the two players score.
We will see how I managed this.

That’s it, we are done for part 1. Next we will see what’s behind the scene.
Valerio