Javascript array performance oddities

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

I’ve recently attended a talk by a Google engineer part of the Google V8 team about writing efficient Javascript code with an eye on performance, obviously with a focus on V8, and I’ve started to read a lot on the topic.
What everyone seems to agree on is that there still is a widespread gap of performances between the different Javascript engines and also sometimes there are odd behaviour that will puzzle you, like the one I’m going to show below.
During the talk I was really impressed by huge performances gap between different approaches to array manipulation, most of the times due to differences in code that looked so trivial!
Before diving in, let’s have a super quick introduction to javascript arrays.

Small overview about Javascript arrays

(Extract from http://goo.gl/2y6oc)
Arrays are one of JavaScript’s core data structures.
Arrays are a special case of objects and they inherit from Object.prototype, which also explains why typeof([]) == “object”.
The keys of the object are positive integers and in addition the length property is always updated to contain the largest index + 1.

Arrays are very useful and used in contexts like games or mathematical simulations for example, where we may need to store and manipulate a lot of objects and we need to do it in a very short amount of time.

State of the art

The general consensus about V8 (and more generally about Javascript engines) has a few useful tips to speed up array manipualtion

  1. If you want to use an array data structure, then treat it as an array: do not mix the types of the data stored. For example in V8 when we first populate the array with data, the JIT creats a HiddenClass object that tracks element types, and if at a certain point we change the types by for example storing a string instead of a number, V8 will have to forget it all and restart, therefore dropping performances.
  2. pre-allocate (pre-allocate means specifying the array length) “small” arrays  in hot portions of the code using the constructor “var array = new Array(num)” or setting the array length if declared with []
  3. do NOT pre-allocate big arrays (e.g. > 64K elements) to their maximum size, instead grow as you go.
  4. it’s best to use WebGL typed arrays (Uint32Array, Float32Array, Float64Array etc…)
  5. use contiguous keys starting at 0 for Arrays
  6. don’t delete elements in arrays, especially numeric arrays
  7. don’t load uninitialized or deleted elements:

I was quite intrigued by the possible performance increase obtainable by simply pre-allocating an array by specifying its size (var array = []; array.length = 1000; or simply var array = new Array(1000) ) so I created a test suite on JsPerf to test these assumptions, and it turns out that Chromes doesn’t really behave as expected, despite being the fastest browser out there anyway.

Test setup:

I setup a two-parts test, as IE 9 doesn’t support typed arrays (and I don’t have access to IE10 yet).
The first part tests the performance differences between pre-allocating and not pre-allocating arrays, and also the difference between array declaration styles
var arr = []
vs
var arr = new Array()
The second part tests typed arrays.

Test part 1 http://goo.gl/oxIwP

Test results: Chrome is the winner

Test results: Chrome is the winner

The figures tell us that Chrome is by far the fastest browser in this test, with Chrome v26 being more ~3x faster than Firefox v17.0.
Let’s now take a look at the assumptions 1 and 2 stated above and see if they are still true when put to the test:
As expected from assumption 1, Chrome is indeed at least 3x faster when pre-allocating a small array (with 1000 items), whereas on Firefox and IE 9 there isn’t any significative difference, with Firefox v17 being more 4x faster than IE 9!
On the browsers tested so far it makes no big difference using the alternative syntax
var arr = new Array(l) vs var arr = [l], although using the constructor is slightly faster on Chrome (especially on Chrome v26) and Firefox.

What about arrays with more than 64k items?
Unsurprisingly Chrome behaves in a different manner than the other two browsers: IE and Firefox do not show any remarkable difference about pre-allocated vs not initialised arrays, with a size of 65000 items. Chrome instead, as expected by assumption 2, performs ~3x faster when the array is not initialised and we are using var arr = []; array initialisation code.
But with much surprise this is not the best performing case, in fact this tests reveals that pre-allocating a big array using the constructor var arr = new Array(65000)  shows that Chrome is ~3x times faster!
Therefore we cannot say assumption 2 is always valid, as we see here that using Array constructor does a lot of difference.
The reason is not clear to me, given that the not-initialised cases (with new Array() and []) perform closely.
And we are not done with the surprises yet!

Test part 2 http://goo.gl/7ouhg
In fact if we take a look at the test about typed arrays, defined here as following

l = window.growFactor;
var arr =  new Uint32Array(l);
  
for (var i = 0; i < l; ++i) {
 arr[i] = 0;
}

shows again Firefox and Chrome performing in a significant different manner: the first benefits a lot by using a typed array (as expected), increasing performance by 2.5x, whereas Chrome sees its performances halved! As a result, Firefox’s performance peek almost matches Chrome’s peek.
Again, it is not clear to me why, I just think this is an oddity that is worth sharing.

Conclusion
We have seen that sometimes browsers performs in an unexpected way (see Chrome with typed arrays), and their performances may differ so much one from the other in things that look trivial and meaningless.
Therefore if your Javascript application is using arrays and needs to squeeze every drop of performance out of the engine, it may be worth spending some time to make some tests as the performance increase can be very significant!

References
http://stackoverflow.com/questions/1246408/use-of-javascript-array-new-arrayn-declaration
http://news.qooxdoo.org/javascript-array-performance-oddities-characteristics
http://www.html5rocks.com/en/tutorials/speed/v8/
https://mkw.st/p/gdd11-berlin-v8-performance-tuning-tricks/#44

C# dynamic object and JSON serialization with Json.Net

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

I’m working on a project where I don’t need a solid domain model as all the logic is handled by the client-side code using  javascript and various frameworks as SignalR, Knockout.js and jQuery.
That’s why I found myself unwilling to add dumb C# POCO objects just to handle JSON serialization and deserialization in the SignalR hub(s), and I turned to dynamic objects and Json.Net (starting from version 4.0)  to handle this tasks easily and quickly.

Serialization

Let’s say that for example I want to send data to the browser. All I have to do is to create an ExpandoObject (my new favourite .Net class!) and serialize it using Json.Net “JsonConvert.SerializeObject()”.

Let’s see an example:

dynamic foo = new ExpandoObject();
foo.Bar = "something";
string json = Newtonsoft.Json.JsonConvert.SerializeObject(foo);

And here is the output:
“{\”Bar\”:\”something\”}”

Deserialization

Json.Net offers a great way to deserialize a JSON string into a dynamic using the JObject (you can find it under the Newtonsoft.Json.Linq namespace and here the details).

Let’s see an example re-using the previous foo object:

dynamic foo = JObject.Parse(jsonText);
string bar = foo.Bar; // bar = "something"

And that’s it, really easy and yet really powerful.
In a more structured application, I would probably not use this trick as I would probably have all of my ViewModels and Domain Models objects ready to use in type safe serialization/deserialization operations, but that’s not always the case.

I hope this helps.

Valerio

My WAT? moment using javascript function parseInt()

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

I know a good developer should always read the manual, but sometimes somebody is  just brave enough to start coding withouth caring to read it all… and that’s how I ended up having my “WAT?” moment using the function parseInt().

I was trying to create a Date object from a json serialized data sent by the server, using parseInt() to transform the day, the month and the year into an int. For several data points the process was successfull, whereas for just a few it was not and I could not understand why… until I pointed Chrome debugger to the right line of code and found out that parseInt has a funny behaviour if we do not specify the radix.

Let me show you what I mean: if you type each of the following line into Chrome developer console, the result you’ll get is what comes after the “–>”

parseInt(“01”) –>1
parseInt(“02”) –>2
parseInt(“03”) –>3
parseInt(“04”) –>4
parseInt(“05”) –> 5
parseInt(“06”) –> 6
parseInt(“07”) –> 7
parseInt(“08”) –> 0
parseInt(“09”) –> 0
WAT?
It comes out that
If the input string begins with “0”, radix is eight (octal). This feature is non-standard, and some implementations deliberately do not support it (instead using the radix 10).  For this reason always specify a radix when using parseInt. (https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/parseInt)
Ok I got it, always specify the radix like parseInt(“08”, 10)  to avoid unexpected results.
I just have one last doubt… how in the world 9 mod 8 is equal to 0? Even Google confirms it’s wrong (try for yourself typing 9 mod 8 in the search box)!

Excel 2010 automation with Asp.Net, on top of a Windows Server 2008 R2 64 with IIS 7.5

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

Anyone who ever tried working with Office automation on Windows Server 2008 64bit knows that it is a nightmare to make it work as expected.
For a web application I’m currently working on, I need to access an Excel 2010 file, write some data in it, run a VBA macro and read the results out of it.
To make this workflow successfull, there are 3 problems to cope with:
1) create an Excel application

var excelApp = new Microsoft.Office.Interop.Excel.Application();

2) open and save files

excelApp.Workbooks.Open(...)

3) run a VBA macro stored into the Excel file

excelApp.Run(...)

Now, problems 1 and 2 have been successfully addressed in the past, see for example http://www.bloing.net/2011/01/how-to-make-iis7-play-nice-with-office-interop/
or
http://social.msdn.microsoft.com/Forums/en-US/innovateonoffice/thread/b81a3c4e-62db-488b-af06-44421818ef91?prof=required
or
http://hopschwiiz.blogspot.fr/2011/02/automating-excel-2007-on-windows-server.html

for a complete overview.

But I couldn’t find anything about problem number 3! My application was simply stuck when trying to run the macro and the only solution was to kill the Excel application to take back control of it. After two days spent troubleshooting, changing every possible configuration options, I’ve finally found the solution. Open IIS Manager, select the Application Pool used for the application you want to deploy and then click on “Advanced Settings”. Once the window opens, set the option “Load User Profile” to true, like shown in the image below.

Interop fix

Set Load User profile to true

That’s it, this was enough to make my application work as expected, being able to run the Excel VBA macro!
I hope this helps.

KnockoutJS data-bind for a set of drop down lists

Standard

Heads up!

The blog has moved!
If you are interested in reading new posts, the new URL to bookmark is http://blog.valeriogheri.com/

 

In my current project I’m building a Single Page Application using Asp.Net MVC 3, jQuery and KnockoutJS.
As part of the UI, I had to build a part of it where the number of dropdown lists associated to a certain product are not known at design time, as the Model is based on an XML resource.
So I was facing the problem of keeping track of all the selected values (and the state of the rest of the UI as well) for each dropdown and without the possibility to use the MVC Model Binding feature because I had to use Ajax (no form submit and thus page re-loading).
After taking a look at what is currently out there to help the developer in this task, I decided to try knockout.js, a nice MVVM javascript framework, to help me keeping track of the state of the UI without writing lot of javascript code.
I’m not going into the concepts behind this framework, neither into details for beginners because they have a great tutorials on their website and nice documentation too, so I couldn’t do better even if I would.

Let’s see a simple example that shows how I solved this problem (I had to ask a question on stackoverflow to seek help as I was stuck. If you are interested, here is the link)

Controller:
Nothing special here, the controller is just injecting some dummy data into the ViewData to fill in the drop down lists and give the illusion that the number of menus is not fixed.

        public ActionResult Index()
        {
            var numberOfDropDownMenus = 5;
            for (var i = 0; i < numberOfDropDownMenus; i++)
            {
                var selectList = new SelectList(this.GetDummyItems(i), "Id", "Name", -1);
                ViewData["Options_" + i] = selectList;
            }
            ViewData["DropDownMenus"] = numberOfDropDownMenus;
            return View();
        }

View:

Let’s start with the knockout view model: for this specific page I’m just interested in the selected value for each ddl, nothing else, so my view model will be just as simple as this:

var initialValues = [];
    var n = @Html.Raw(Json.Encode(ViewData["DropDownMenus"]));
    for (var i = 0; i < n; i++) {
        initialValues.push(ko.observable());
    }

var viewModel = {
        selectedValues: ko.observableArray(initialValues)
};

The above code means that our view model has only one item, selectedValues, which is an observable array of observables (initialValues is in fact an array of observables).

Why not simply use an observableArray? Because we want each item of the array to reflect the currently selected value for a specific DDL, and therefore it must be an observable value so that knockout can automatically update its value whenever the user changes something.

It will be clearer if we look at how we data-bind each DDL to an observable value into the observable array:

@for (var i = 0; i < (int)ViewData["DropDownMenus"]; i++)
{
    @Html.DropDownList("DropDown_" + i, (SelectList)ViewData["Options_" + i],
                                new { data_bind = "value: selectedValues()[" + i + "]" })
}

The trick here is to use a positional databinding:

new { data_bind = “value: selectedValues()[” + i + “]” }

If i is 0, this will be evaluated for example as
new { data_bind = “value: selectedValues()[0]” }

The currently selected value for the first DDL will be stored in the first position of the observable array!

User interface and test output

User interface and test output

In the screenshot above we can see that the array is changing accordingly with the changes applied to the UI.
(To show the test output, I used the following syntax: <div data-bind=”text: ko.toJSON(selectedValues)”></div> It’s important to use ko.toJSON since selectedValues contains observables, not simple values)

Then, if we want to send data to the server (and we want), we can do like the following

$(document).ready(function () {
        $("#fakePost").click(function () {
            var viewModelToJSON = ko.toJSON(viewModel); //Serialize the viewmodel
           $.post("/Home/MyAction", { JSONviewModel: viewModelToJSON }, myCallback);
        });
        ko.applyBindings(viewModel);
    });

And on the server we can deserialize the string using our own favourite library.
For the sake of simplicity I used the built in javascript serializer:

public JsonResult MyAction(string JSONviewModel)
        {
            var jss = new JavaScriptSerializer();
            var selectedValues = jss.Deserialize(JSONviewModel);
            return Json("Ok");
        }

where ViewModel is a simple C# object useful for easy de-serialization and to communicate with an hypothetic Application Layer

public class ViewModel
    {
        public List selectedValues { get; set; }
    }

That’s all!
Obviously this is a super simplified example, but I hope it helps if someone has to face the same problem as me in the future.

Cheers,

Valerio