Telerik blogs
javascript_testing_header

Having tests is important. They allow us to extend and refactor our code with ease. Many developers follow test driven development as a workflow. I believe that writing tests makes software development much more interesting and generally leads to better code. Well-designed and tested systems are easier to maintain.

Over the last few years, developers have begun to put a lot of application logic in the browser. We started writing more and more JavaScript. And because the language became so popular, developers started creating tools designed to make the experience of JavaScript development better. In this article, we will cover some of those tools specifically designed for testing client-side JavaScript code.

The Testing Setup

Let's discuss the types of tools that make testing possible. They enable us to build, group and run our tests.

Testing framework

The framework consists of functions like suite, describe, test or it. These give us an ability to create groups of tests. These groups are often called suites. For example:

describe('Testing user management', function () {
  it('should register a new user', function(done) {
    // the actual testing goes here
  });
  it('should remove a user', function(done) {
    // the actual testing goes here
  });
});

We split the logic of our application in blocks. Every block has its own suite that includes related tests that we want to run against our code.

Popular testing frameworks for JavaScript are QUnit, Jasmine or Mocha.

Assertion Library

We use assertion libraries to perform the actual checks. They provide simple to use functions like in the following example:

expect(5).to.be.a('number');

There are plenty of modules that we can use. Node.js even has a built-in one. Chai, Expect or should.js are some of the open source options at our disposal.

I should mention that some testing frameworks come with their own assertion library.

Runner

We may or may not need a runner. In some cases, just a testing framework is not enough and we need to way to run the tests within a specific context. To do that we use a runner. In some cases these tools are called "spec runners" or "test runners." These instruments wrap our test suites and run them in a special environment, which we will cover in a bit.

The Application

We need an application that needs testing. Although this is for illustration purposes, it shouldn't be too simple. The TODOMVC projects seems like a good choice. It's basically the same TODO app built with a variety of different frameworks. Let's use the Backbone.js variant. This is what the application looks like:

todomvc application

Let's imagine that this is a project that we built over the last month and we plan to release it next week. We want to make sure it is covered with a couple of tests. Let's also imagine that the back-end is already tested; the only one part that is questionable is our client-side JavaScript. At this point, since the application is already done we are mostly interested if it does its job. It is a TODO app, so we need to make sure that the user is able to add, delete and edit tasks.

Testing in the browser

We need to execute checks against code that runs in the browser, so it makes sense to place our tests there. We are going to use Mocha as a framework and, because that framework comes without an assertion library, we will bring Chai to the project. We download the app from todomvc.com and browse the files:

├── bower_components
├── js
│   ├── collections
│   ├── models
│   ├── routers
│   ├── views
│   └── app.js
└── bower.json
└── index.html
└── readme.md

This article is about testing so we are not going into details on how Backbone.js works. However, here's some basic details about the files and directories:

  • bower_components - contains the Backbone.js library, a localStorage helper, jQuery, Underscore.js and the TODOMVC common files;
  • js - this directory contains the actual code of the app;
  • bower.json - defines the project's dependencies;
  • index.html - contains the HTML markup (templates).

In the usual development workflow we open a browser, load the application and use the UI. We add a new TODO by typing something in the input field, press the Enter key and the task appears below. Removing records works by clicking on the small X symbol. Editing is possible via double-clicking on the TODO. We have a good number of actions involving different mouse events.

In order to test the app, we don't want to have to repeat the steps above over and over again. The automated test should take care of that for us. Effectively, we will run the application and will write code that interacts with the page in much the same as we would do it manually. Let's create a new folder tests_mocha where we can place our tests.

├── bower_components
├── js
│   ├── collections
│   ├── models
│   ├── routers
│   ├── views
│   └── app.js
├── tests_mocha
│   ├── package.json
│   ├── spec.js
│   └── TestRunner.html
└── bower.json
└── index.html
└── readme.md

Both, Mocha and Chai, are installable via npm, which comes with Node.js. We simply add them in a package.json file and run npm install within the same directory.

// package.json
{
  "name": "project",
  "version": "0.0.1",
  "devDependencies": {
    "chai": "1.10.0",
    "mocha": "2.1.0"
  }
}

Creating the Test Runner

spec.js will contain our test and TestRunner.html is a copy of index.html that we will modify. We need the application running so we should definitely use the code in index.html. Let's see how the original file looks before we discuss the changes:

<!doctype html>
<html lang="en" data-framework="backbonejs">
  <head>
    <meta charset="utf-8">
    <title>Backbone.js • TodoMVC</title>
    <link rel="stylesheet" href="bower_components/todomvc-common/base.css">
  </head>
  <body>
    <section id="todoapp">
      ... the base markup of the application
    </section>
    <footer id="info">
      ... the markup of the footer
    </footer>
    <script type="text/template" id="item-template"> ... </script>
    <script type="text/template" id="stats-template"> ... </script>

    <script src="bower_components/todomvc-common/base.js"></script>
    <script src="bower_components/jquery/dist/jquery.js"></script>
    <script src="bower_components/underscore/underscore.js"></script>
    <script src="bower_components/backbone/backbone.js"></script>
    <script src="bower_components/backbone.localStorage/backbone.localStorage.js"></script>
    <script src="js/models/todo.js"></script>
    <script src="js/collections/todos.js"></script>
    <script src="js/views/todo-view.js"></script>
    <script src="js/views/app-view.js"></script>
    <script src="js/routers/router.js"></script>
    <script src="js/app.js"></script>
  </body>
</html>

Client-side apps can get really complex and the one that we have needs a couple of things to run. All the <script> tags at the end of file need to stay including the <script> tags representing the templates. The <section> is important and we will also keep it.

In order to see the results of the tests we have to add the Mocha's styles. We are testing only JavaScript so the TODOMVC's base.css may be removed. Replace the <link> tag in the head with:

<link rel="stylesheet" href="./node_modules/mocha/mocha.css">

The styles are gone but the markup is still there. However, we do not need it to be visible, so let's add an additional CSS class that hides it:

<style type="text/css">
  #todoapp, #info {
    display: none;
  }
</style>

With these two changes we still have the project running and works, but its interface is not visible. We have a blank page waiting for our tests. The code below goes at the end of the page. Just after TODOMVC's scripts:

<div id="mocha"></div>
<script src="./node_modules/mocha/mocha.js"></script>
<script src="./node_modules/chai/chai.js"></script>
<script>
    mocha.ui('bdd'); 
    mocha.reporter('html');
    var expect = chai.expect;
</script>
<script src="./spec.js"></script>
<script>
    mocha.run();
</script>

The empty <div> is used by the framework for showing the results. After that we add Mocha and Chai. Lastly, we run the tests. Notice that our spec.js is added when the testing framework and the assertion library are initialized and before mocha.run(). In practice, the browser:

  • Imports Mocha's CSS;
  • Imports TODOMVC's files and runs the application;
  • Imports Mocha and Chai;
  • Imports our spec.js;
  • Runs the tests.

Writing the Tests

So far so good. Let's write our test. We mentioned earlier that we want to check three things: is the user able to add, delete and edit TODOs.

// spec.js
describe("Testing TODOMVC", function () {

  var setText = function(text, selector) {
      var input = $(selector || '#new-todo');
      var e = $.Event("keypress");
      e.which = e.keyCode = 13;
      return input.val(text).trigger(e);
  };

  before(function() {
      window.localStorage.removeItem('todos-backbone', '');
      app.todos.reset();
  });

  it("Adding new TODOs", function () {
      setText('TODO A');
      setText('TODO B');
      expect($('#todo-list li').length).to.be.equal(2);
  });
  it("Deleting TODO", function () {
      $('#todo-list li:first-child .destroy').click();
      expect($('#todo-list li').length).to.be.equal(1);
  });
  it("Edit and add TODOs", function () {
      setText('A new TODO');
      $('#todo-list li:first-child').addClass('editing');
      setText('A new TODO', '#todo-list li:first-child .edit').blur();
      expect($('#todo-list li').length).to.be.equal(2);
      expect($('#todo-list li label').eq(0).text()).to.be.equal($('#todo-list li label').eq(1).text())
  });

});

The test starts with a describe call, wrapping our checks in a test suite. The setText function is a helper that changes the value of an input field and simulates the pressing of the Enter key.

Most testing frameworks allow us to execute logic before the tests run. That is what we are doing with the before function. In our case we need to clear the data saved in localStorage because later we expect to see a specific number of TODOs in the list.

The following it calls represent the three operations of adding, removing and editing. Notice that we used $ (jQuery) and expect which are global items.

Running Tests

With this code complete, we can open TestRunner.html in our favorite browser and the result should be:

mocha testing

We can now guarantee that our project provides the required functionality. We have a test and it is kind of automated. We say kind of because the test is run in a browser and we still have to open TestRunner.html manually.

That's one of the issues with this approach. We can't force the developers to run the tests in their browsers all the time. The testing process should be part of the deployment or committing process. In order to achieve this we need to move our tests into the terminal.

Testing with PhantomJS

The next problem that we have to deal with is finding a browser that works in the terminal. We need a browser because Backbone.js needs to render the UI and we need to interact with it. There are a specific type of browsers called headless browsers which do not have a visible interface.

We control them via code. However, they are fully functional same as the real browsers. One of the most popular ones is PhantomJS. Let's try it out and see how it will handle our test.

PhantomJS is distributed as an executable file. Or in other words, after successful installation we have a phantomjs command available. It is available for pretty much every operating system. Along with headless browser we are going to use Node.js and its package manager (npm).

Running Tests in the Terminal

Assuming that we have PhantomJS installed, the next step is running our Mocha tests via the terminal. In practice, we have to load TestRunner.html inside the browser and check the results from the Mocha framework. We may do that manually but to save some time we will use a Node.js module called mocha-phantomjs.

A quick run of npm install -g mocha-phantomjs will make mocha-phantomjs available in our console. Directly copy the tests_mocha folder into a new directory tests_mocha-phantomjs. The only change that we have to make is changing:

<script>
  mocha.run();
</script>

...to:

<script>
  if (window.mochaPhantomJS) { mochaPhantomJS.run(); }
  else { mocha.run(); }
</script>

The module gets the respose from the testing framework and sends it to the terminal. Here is what the result should look like:

mocha-phantomjs

Now our checks run into the terminal and we can add the mocha-phantomjs call to our continuous integration setup.

PhantomJS is nice but it runs our code in a single browser. What if we need to test our code against different browsers. Especially with client-side JavaScript, we should make sure that our applications work in different environments. And not only that, we want to test in real browsers. Karma is a project that provides this functionality.

Using Karma as a test runner

Similar to mocha-phantomjs, Karma is distributed as a Node.js module. However, this time we will need not one but a couple of modules. So, let's follow the same idea as above - copy tests_mocha to a new folder tests_karma. The package.json file should look like that:

{
  "name": "project",
  "version": "0.0.1",
  "devDependencies": {
    "karma": "https://github.com/krasimir/karma/tarball/589f55a8abab613ec915a871ee976ca0e15ad36f",
    "karma-mocha": "^0.1.10",
    "karma-phantomjs-launcher": "^0.1.4",
    "karma-chrome-launcher": "0.1.7",
    "karma-chai": "0.1.0"
  }
}

In addition to the package above, we will also need karma-cli. Run npm install -g karma-cli after you run npm install (which will install the dependencies listed above). karma-cli has the karma command that we can call to run the test runner from the terminal.

The Trouble with Karma

I'd read a lot about Karma over the last few months and really wanted to try it. However, I found that it is designed more for unit testing and not integration (journey) testing.

The command line tool of the runner accepts configuration in the form of a JSON object. There are several options, but none of them accept a HTML file. We may send JavaScript files to the browser, but we can't define the HTML that will be loaded.

There is a context template that Karma uses. It injects our JavaScript and tests and that's all. For our use case, this is not enough. We have HTML markup and templates in <script> tags. They have to be in the page before running the application.

The Solution

What I did is to fork the project and made a little change giving one more option in the configuration file. This gives us a way to set our own context template, which is why I added a URL to my fork in the package.json file.

Keeping the same spec.js the same, I had to change TestRunner.html to:

<!DOCTYPE html>
<html>
<head>
  <title></title>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
</head>
<body>
  <section id="todoapp"> ... </section>
  <footer id="info"> ... </footer>
  <script type="text/template" id="item-template"> ... </script>
  <script type="text/template" id="stats-template"> ... </script>

  <script type="text/javascript">
    if (window.opener) {
      window.opener.karma.setupContext(window);
    } else {
      window.parent.karma.setupContext(window);
    }
    %MAPPINGS%
  </script>
  %SCRIPTS%
  <script type="text/javascript">
    window.__karma__.loaded();
  </script>
</body>
</html>

The needed <section> and <footer> are included along with the templates. At the end is the code that Karma uses for constructing the final document and running the tests.

Configuration

I mentioned that the framework uses a configuration file. Create a file named tests_karma/karma.conf.js with the following content:

// karma.conf.js
module.exports = function(config) {
  config.set({
    basePath: '',
    client: {
      contextFile: '/TestRunner.html'
    },
    frameworks: ['mocha', 'chai'],
    files: [
      '../bower_components/todomvc-common/base.js',
      '../bower_components/jquery/dist/jquery.js',
      '../bower_components/underscore/underscore.js',
      '../bower_components/backbone/backbone.js',
      '../bower_components/backbone.localStorage/backbone.localStorage.js',
      '../js/models/todo.js',
      '../js/collections/todos.js',
      '../js/views/todo-view.js',
      '../js/views/app-view.js',
      '../js/routers/router.js',
      '../js/app.js',
      'spec.js'
    ],
    exclude: [ ],
    preprocessors: { },
    reporters: ['progress'],
    port: 9876,
    colors: true,
    logLevel: config.LOG_INFO,
    autoWatch: false,
    browsers: ['PhantomJS', 'Chrome'],
    singleRun: false
  });
};

Within this configuration, we list the JavaScript files that need to be injected into the page. The contextFile setting is the change that I discussed earlier. Notice that we have specified two browsers for our tests will run in.

Running Karma

The final step is running karma start ./karma.conf.js --single-run in the terminal. Here is the result:

karma

In the beginning of this section we added karma-phantomjs-launcher and karma-chrome-launcher. The framework uses these two modules to run the browsers that we specified.

So, we tried running the tests in the browser but this approach does not scale. We've run them in the terminal via mocha-phantomjs but that means testing in only one browser. The third try was using Karma as a runner and lanuching two different browsers one of which was not headless. The last setup was kind of complex, involving several modules and even a framework patch. Let's try it with another runner - DalekJS.

Testing with DalekJS

DalekJS uses a different approach. It doesn't require a testing framework or assertion library. It has its own.

The installation is pretty easy. We again need Node.js and npm because the tool is distributed as a Node.js package. Similar to Karma, we also need a CLI client and we also need the DalekJS framework itself.

// after this line we will have `dalek` command available
npm install dalek-cli -g

Let's create another folder called tests_dalekjs containing the following package.json file:

{
  "name": "project",
  "version": "0.0.1",
  "dependencies": {
    "dalekjs": "0.0.9"
  }
}

When we have the both modules installed we may proceed with writing our test.

Writing the Test

The good news is that we don't have to touch the HTML at all. We can copy our index.html to tests_dalekjs/TestRunner.html and just correct the paths to the JavaScript files - the rest is the same.

Because DalekJS has its own systax we can't use the spec.js file from the tests above. Here is how our three operations look written with the DalekJS API:

var setText = function(text, selector) {
  var input = $(selector || '#new-todo');
  var e = $.Event("keypress");
  e.which = e.keyCode = 13;
  input.val(text).trigger(e);
};

module.exports = {
  'Testing TODOMVC app': function (test) {
    test
    .open('TestRunner.html')
    .execute(setText, 'TODO A')
    .execute(setText, 'TODO B')
    .assert.numberOfElements('#todo-list li', 2, 'Should have two TODOs')
    .execute(function() {
      $('#todo-list li:first-child .destroy').click();
    })
    .assert.numberOfElements('#todo-list li', 1, 'Should have two TODOs')
    .execute(setText, 'A new TODO')
    .execute(function() {
      $('#todo-list li:first-child').addClass('editing');
      $('#todo-list li:first-child .edit').val('A new TODO').blur();
    })
    .assert.numberOfElements('#todo-list li', 2, 'Should have two TODOs')
    .assert.text('#todo-list li:first-child label').is('A new TODO')
    .assert.text('#todo-list li:second-child label').is('A new TODO')
    .done();
  }
};

Again we have a helper method for setting the value of an input field and triggering the Enter button press. The object that we export contains all the actions needed for adding, removing and editing TODOs. It is nice to see that the API is designed this way as it clear what is going on just by reading the methods' execution.

The only method that is a little bit tricky is execute. It accepts a function that will be executed in the context of the browser.

Running the Test

We can run the test using the command dalek ./tests_dalekjs/spec.js. The result should be:

DalekJS

We should keep in mind that DalekJS, similar to Karma, can run Chrome, IE, Firefox and Safari. Pretty much every modern browser is supported. All we have to do is to install additional modules like dalek-browser-chrome for example. More about the supported browsers can be found here.

Atomus - One More Tool for Testing

Atomus is the tool that I use at work. All the options above are great. Most of them are well tested with big communities. However, in my view, they are not ideal.

It is great that we can cover user journeys, but very often we want to test only part of the whole application. What if we want to test only a particular view of our Backbone.js app. With DalekJS this is really difficult. With Karma it is possible but kind of tricky. The mocha_phantomjs module is close to a working solution but it has some limitations.

While we were thinking about unit testing we realized that all we need is a DOM simulation. In most cases we are not interested waht the UI looks like, just how it behaves. Then we found jsdom, which is a JavaScript implementation of the WHATWG DOM and HTML standards.

We created some tests and found out that it works pretty well. It supports DOM manipulations and DOM event dispatching/listening. Even Ajax requests are supported. Atomus is a wrapper around jsdom providing a more robust and friendly API.

Creating the Test

Let's finish this article by covering the TODOMVC testing with Atomus. We will again use Mocha and Chai. Create a package.json in a new directory called tests_atomus and define the dependencies:

{
  "name": "project",
  "version": "0.0.1",
  "devDependencies": {
    "chai": "1.10.0",
    "mocha": "2.1.0",
    "atomus": "0.1.12"
  }
}

TestRunner.html is almost the same as the original index.html file - only the importing of the external JavaScript files should be removed as this will happen in the spec.js file. That file starts with the following code:

var fs = require('fs');
var expect = require('chai').expect;

describe("Testing TODOMVC", function () {

  var atomus = require('atomus');
  var htmlStr = fs.readFileSync('./TestRunner.html').toString('utf8');
  var browser, $;

});

We use the file system API to read the content of the TestRunner.html. The browser variable will represent the Atomus API. The library automatically injects jQuery so $ will be our shortcut. The first test looks like:

it("Adding new TODOs", function (done) {
  browser = atomus()
  .external(__dirname + '/../bower_components/todomvc-common/base.js')
  .external(__dirname + '/../bower_components/jquery/dist/jquery.js')
  .external(__dirname + '/../bower_components/underscore/underscore.js')
  .external(__dirname + '/../bower_components/backbone/backbone.js')
  .external(__dirname + '/../bower_components/backbone.localStorage/backbone.localStorage.js')
  .external(__dirname + '/../js/models/todo.js')
  .external(__dirname + '/../js/collections/todos.js')
  .external(__dirname + '/../js/views/todo-view.js')
  .external(__dirname + '/../js/views/app-view.js')
  .external(__dirname + '/../js/routers/router.js')
  .external(__dirname + '/../js/app.js')
  .html(htmlStr)
  .ready(function(errors, window) {
    $ = window.$;
    browser.keypressed($('#new-todo').val('TODO A'), 13);
    browser.keypressed($('#new-todo').val('TODO B'), 13);
    expect(window.$('#todo-list li').length).to.be.equal(2);
    done();
   });
});

This is the place where we initialize our virtual browser, define the external JavaScript files and set the HTML markup of the page. The ready function accepts a callback that is fired once the page and its resources are fully loaded. We receive a window object, which is the same window object that we have in a real browser. We can think of that variable as a pointer to all the browser APIs and of course to the global scope.

Atomus has methods like keypressed and clicked that simulate user interactions. We can also use the popular jQuery methods for achieving the same result but these built-in methods solve some nasty bugs.

Now that we have our browser available, we can continue with deleting and editing TODOs.

it("Deleting TODO", function () {
  $('#todo-list li:first-child .destroy').click();
  expect($('#todo-list li').length).to.be.equal(1);
});

it("Edit and add TODOs", function () {
  browser.keypressed($('#new-todo').val('A new TODO'), 13);
  $('#todo-list li:first-child').addClass('editing');
  $('#todo-list li:first-child .edit').val('A new TODO').blur();
  expect($('#todo-list li').length).to.be.equal(2);
  expect($('#todo-list li label').eq(0).text()).to.be.equal($('#todo-list li label').eq(1).text())
});

Running the Tests

To run the tests we don't need a CLI client; we just have to type mocha ./spec.js and the result is:

Atomus

Notice that when we are using Atomus the main thing is not the runner or the headless browser - it's the testing framework. That is what drives the tests. Atomus is just a helper tool.

In our case this helped us cover different levels of the client-side architecture. We have tests for simple UI elements and at the same time use the same tool for integration testing.

To finish with something interesting let's look at the following code:

var mockups = [
  { 
    url: '/api/method/action',
    method: 'GET',
    response: {
      status: 200,
      responseText: JSON.stringify({"id": "AAA"})
    }
  }
];

var atomus = require('../lib');
var b = atomus()
.ready(function(errors, window) {
  b.addXHRMock(mockups);

  var $ = window.$;
  $.ajax({
    url: '/api/method/action'
  }).done(function(result) {
    console.log(result.id); // AAA
  });

});

In a complex environment, where the application's UI communicates with the backend, we need mocking of HTTP requests. In our case, we wanted to cover a complete user journey but this was complicated by the fact that we wanted to have the tests independent. So, we added a mock of the traditional XMLHttpRequest object and now the developer is able to link URLs with particular responses.

Conclusion

Testing is fun - especially when we have so many tools to play with. No matter what kind of application we have, we should know that there is way to test it. I hope that this article helps you find the right tool that fits your project.

Header image courtesy of Manoj Vasanth


Telerik Blogging Ninja
About the Author

Krasimir Tsonev

Krasimir Tsonev is a front-end developer, blogger and speaker. He loves writing JavaScript and experimenting with the latest CSS and HTML features. Author of the Node.js blueprints book, he is focused on delivering cutting edge applications. Right now, with the rise of the mobile development, he is enthusiastic to work on responsive applications targeted to various devices. Living and working in Bulgaria he graduated at the Technical University of Varna with bachelor and master degree in computer science. You can find him on GitHub here.

Comments

Comments are disabled in preview mode.