JavaScript continuous testing with QUnit, PhantomJS and Powershell

I’ve started to write more JavaScript code lately, and since the software development process I follow most is TDD I have been using QUnit used by the famous JQuery project.

Writing tests with qunit is very simple;

test("a basic test example", function() {
  ok( true, "this test is fine" );
  var value = "hello";
  equal( value, "hello", "We expect value to be hello" );
});

with some small differences we can say that QUnit is the *Unit framework for javascript (JUnit, NUnit, etc.).

In order to run your tests suite, you need to write an htm document as explained in the qunit website. You will normally open the page from several browsers in order to run the tests with different JavaScript engines.

PhantomJS

PhantomJS is a command-line tool that packs and embeds WebKit. It comes for windows, mac and linux. The windows version is a single exe file called “phantomjs.exe”.

If you google PhantomJS + Qunit you will find several scripts and most of them are outdated and will not run with the current version of PhantomJS (1.2.0 at the moment). I am using this script which is an slightly modified version of one that I found;

/**
 * Wait until the test condition is true or a timeout occurs. Useful for waiting
 * on a server response or for a ui change (fadeIn, etc.) to occur.
 *
 * @param testFx javascript condition that evaluates to a boolean,
 * it can be passed in as a string (e.g.: "1 == 1" or "$('#bar').is(':visible')" or
 * as a callback function.
 * @param onReady what to do when testFx condition is fulfilled,
 * it can be passed in as a string (e.g.: "1 == 1" or "$('#bar').is(':visible')" or
 * as a callback function.
 * @param timeOutMillis the max amount of time to wait. If not specified, 3 sec is used.
 */
function waitFor(testFx, onReady, timeOutMillis) {
    var maxtimeOutMillis = timeOutMillis ? timeOutMillis : 3001, //< Default Max Timout is 3s
        start = new Date().getTime(),
        condition = false,
        interval = setInterval(function() {
            if ( (new Date().getTime() - start < maxtimeOutMillis) && !condition ) {
                // If not time-out yet and condition not yet fulfilled
                condition = (typeof(testFx) === "string" ? eval(testFx) : testFx()); //< defensive code
            } else {
                if(!condition) {
                    // If condition still not fulfilled (timeout but condition is 'false')
                    console.log("'waitFor()' timeout");
                    phantom.exit(1);
                } else {
                    // Condition fulfilled (timeout and/or condition is 'true')
                    console.log("'waitFor()' finished in " + (new Date().getTime() - start) + "ms.");
                    typeof(onReady) === "string" ? eval(onReady) : onReady(); //< Do what it's supposed to do once the condition is fulfilled
                    clearInterval(interval); //< Stop this interval
                }
            }
        }, 100); //< repeat check every 250ms
};


if (phantom.args.length === 0 || phantom.args.length > 2) {
    console.log('Usage: run-qunit.js URL');
    phantom.exit(1);
}

var page = new WebPage();

// Route "console.log()" calls from within the Page context to the main Phantom context (i.e. current "this")
page.onConsoleMessage = function(msg) {
    console.log(msg);
};

page.open(phantom.args[0], function(status){
    if (status !== "success") {
        console.log("Unable to access network");
        phantom.exit(1);
    } else {
        waitFor(function(){
            return page.evaluate(function(){
                var el = document.getElementById('qunit-testresult');
                if (el && el.innerText.match('completed')) {
                    return true;
                }
                return false;
            });
        }, function(){
            var failedNum = page.evaluate(function(){

                var tests = document.getElementById("qunit-tests").childNodes;
                console.log("\nTest name (failed, passed, total)\n");
                for(var i in tests){
                    var text = tests[i].innerText;
                    if(text !== undefined){
                        if(/Rerun$/.test(text)) text = text.substring(0, text.length - 5);
                        console.log(text + "\n");    
                    }
                }

                var el = document.getElementById('qunit-testresult');
                console.log(el.innerText);
                try {
                    return el.getElementsByClassName('failed')[0].innerHTML;
                } catch (e) { }
                return 10000;
            });
            phantom.exit((parseInt(failedNum, 10) > 0) ? 1 : 0);
        });
    }
});

Then, in order to run your tests you have to execute this from the command-line:

phantomjs.exe run-qunit.js path\to\your_qunit.htm

where run-qunit.js is the above script. This will run all your tests and give important information as if you where running them from a WebKit browser (Safari, Chrome, etc.). This doesn’t means your tests are okay for other browsers but it is a good start.

Continuous testing

Now that we can run our tests suite from the console and given that JavaScript is an interpreted language (i.e. we don’t have to wait the compiler ;) ) my question was if we can run those tests continuously.

Continuous testing is something beyond continuous integration; the methodology is simple; there is something watching for your source code files, when you save the files the program will run of all your tests ins background. This technique is often used in dynamic languages; for instance in the Ruby ecosystem is Autotest.

Powershell script

My continuous testing tool in windows, is this pretty simple PowerShell script;

$m = Get-Module -List pswatch 
if(!$m) {
    Write-Host "Uh-Oh you don't have the pswatch cmdlet installed. Let me handle that for you."
    iex ((new-object net.webclient).DownloadString("http://bit.ly/Install-PsWatch"))
} 

Import-Module pswatch

watch | %{
    write-host "Change in $_.Path"
    phantomjs.exe run-qunit.js $_.Path
}

This script uses a CmdLet I wrote called PsWatch. It will auto-install the CmdLet if you don’t already have it.

Other scripts

If you are working on a Linux box, instead of Powershell and PsWatch you can use ruby watchr.


blog comments powered by Disqus
  • Categories

  • Archives