Automated Regression Testing Made Easy with CasperJS

One hot topic in today’s web test automation is regression testing. A regression is a bug introduced into a system after certain changes have been made.

These changes can include patching, new functionality development, integration of third-party components, and so on.

In agile development, regression testing is critically important to make sure your product still functions properly at the end of every iteration or sprint. Automating your regression tests means you can run them faster and more frequently, while eliminating human error during test execution.

In this article, we will explore writing regression tests with CasperJS, an easy-to-use JavaScript testing framework that leverages PhantomJS.

PhantomJS? CasperJS?

PhantomJS is a fast, scripted, headless WebKit-based browser — ideal for automating web page interaction. PhantomJS has excellent support for many Web standards, including DOM manipulation, CSS3 selectors, Canvas, AJAX, and more.

CasperJS is a powerful utility that runs on top of PhantomJS.

Casper provides a raft of high-level methods that greatly facilitate things like navigating web pages, clicking buttons, filling forms, and downloading resources. In addition, it includes a nice testing framework — compete with JUnit support for easy CI integration.

CasperJS has been around since 2011, so it’s a solid tool with a well-defined API and documentation that is lucid, concise, and generally a pleasure to read.

Getting Started

Installation

Neither PhantomJS nor Casper are Node.js modules, but they can still be conveniently installed with npm:

npm install -g phantomjs
npm install -g casperjs

Organizing tests

A nice way to organize your CasperJS code is to put each test suite into a separate JavaScript file. That way, each script can focus on a specific portion or functionality slice of your website, such as homepage, search, or contact form.

The tester module

Casper ships with a Tester class and a tester module. The module provides a unit and functional testing API that is a great fit for our purposes.

An instance of the class can be accessed via the test property of any Casper class instance. The test object allows us to run assertions over the current context, pass or fail tests, log errors, and more.

Your typical test suite utilizing the test object and residing in a single *.js file must be run with the test subcommand like so:

casperjs test /path/to/your/test.js

Testing the Homepage

By way of gentle introduction, let’s look at Oxagile’s homepage and see what kind of things we could check for possible regressions whenever we update the content or codebase.

First off, we need to make sure our homepage actually loads and returns a 200 status code. We would also do well to check the page title for important keywords that must be there.

So let’s write a minimal test suite and store it in a file called tests.homepage.js:

casper.test.begin('Homepage Tests', 2, function suite(test) {
    casper.start('http://oxagile.com', function() {
        test.assertHttpStatus(200, '01 - Homepage must be up and running.');
        test.assertTitleMatch(/^Custom Software Development Company/, '02 - Homepage title must contain the right keywords.');
    });
    casper.run(function() {
        test.done();
    });
});

Several things are happening here:

1. the begin() method starts our test suite, with 2 being the expected number of tests to run;
2. the test suite will open a URL using the start() method and execute a function after the URL is loaded;
3. this function is where the actual testing takes place; we’ll add more tests to it as we go along;
4. run() runs the whole test suite and executes a callback when it finishes (the done() method simply flags the suite as complete).

OK, we are ready to run our script:

casperjs test tests.homepage.js

You should get the following output in your terminal:

Now we will expand our test suite by adding more assertions. Let’s make sure that:

1. the above-the-fold area actually contains the proper contact details (phone number and email, both important);
2. the navigation menu has all five top-level items in place;
3. the floating Free Quote button is present;
4. the six client logos are all visible;
5. the three main service offerings are there and contain the right keywords;
6. the number of technology icons at the bottom is seven.

Adding these tests is easy with the assertExists(), assertSelectorHasText(), and assertElementCount() methods:

test.assertSelectorHasText('div.phone', '+1 855 466 9244', '03 - Phone number must be correct.');
test.assertSelectorHasText('a.mail', 'contact@oxagile.com', '04 - Email must be correct.');
test.assertElementCount('div.inner-wrapper ul.prime > li', 5, '05 - Five top-level menu items must exist.');
test.assertExists('div.free-quote-btn', '06 - "Free Quote" button exists.');
test.assertElementCount('div.clients > div > img', 6, '07 - Six client logos must exist.');
test.assertSelectorHasText('div.soft', 'Custom Software Development', '08 - Custom Software Development service must exist.');
test.assertSelectorHasText('div.web', 'Web Application Development', '09 - Web Application Development service must exist.');
test.assertSelectorHasText('div.mobile', 'Mobile Application Development', '10 - Mobile Application Development service must exist.');
test.assertElementCount('div.technologies div.list img', 7, '11 - Seven tech expertise icons must exist.');

Running the above code should produce the following result:

As you can see, it took us literally a couple of seconds to ensure that our homepage displays most of the information that we care about.

Let’s go further and check if the search form works.

For filling forms, CasperJS provides a handy method aptly called fill(). The method takes three parameters:

1. a CSS selector to identify the form,
2. an object for input field name and value pairs,
3. and a boolean parameter that determines whether the form is automatically submitted.

By way of testing, let’s search for test automation and count the results. Fill the search field and submit the form:

casper.fill('form#searchform', {
    's': 'test automation'
}, true);

We now wait till the results page loads so we can run our test. We’ll also print the search results to console as an extra check:

casper.waitForUrl(/\?s=.+/, function() {
    test.assertElementCount('div.search-list-item-title', 10, '12 - Ten search results must be displayed.');
    console.log('\nSearch results:');
    this.getElementsInfo('div.search-list-item-title').forEach(function(_item, _index) {
        console.log(_index + 1 + '. "' + _item.text + '"');
    });
    console.log('');
});

The output should look like this:

In the above code snippet, getElementsInfo() is a convenience method that retrieves data for all elements matching a CSS selector and returns an array that contains an object representation of elements. We use its text key to retrieve search result titles.

Testing Portfolio Images

Now let’s do something a bit more difficult. Each project in our portfolio has a gallery of images associated with it:

Let’s walk through all the portfolio projects and make sure that:

1. the main (top) portfolio image is there;
2. three additional images are present;
3. all the images have correct dimensions, namely 600x480px.

This script will rely on Casper’s evaluate() method. The method allows you to evaluate an expression within the context of the current DOM.

This is an important concept to grasp when working with PhantomJS or Casper: evaluate() acts as a bridge between the casperjs environment and page context. Simply put, when you pass a function to evaluate(), it will be executed as if you typed it into the browser’s console.

Using evaluate() allows us to enter the DOM, run some JS code, and return values for further processing within the Casper environment. Which is exactly how we are going to get our gallery image sizes so we can compare them and verify the dimensions.

For this example, let’s assume you have a list of URLs stored in urls.txt.

Obtaining URLs from a website could be easily accomplished with tools like wget (man wget and check out its --spider option). We are going to load the list of URLs into an array and process them one by one.

Here’s the complete code of the portfolio test suite:

var fs = require('fs');
var aUrls = [];
stream = fs.open('urls.txt', 'r');
line = stream.readLine();
var i = 0;
while (line) {
    aUrls.push(line);
    line = stream.readLine();
    i++;
}
casper.test.begin('Portfolio Tests', aUrls.length * 6, function suite(test) {
    casper.start();
    casper.then(function() {
        // walk through the array:
        aUrls.forEach(function(sUrl, index) {
            // open URL:
            casper.thenOpen(sUrl);
            // wait for the URL to load:
            casper.waitForUrl(sUrl, function() {
                iNum = index + 1 + ' ';
                this.echo("\nTesting URL " + sUrl);
                test.assertElementCount('img.portfolio-main-image', 1, iNum + '[1] Page must have one main portfolio image.');
                test.assertElementCount('img.portfolio-additional-image', 3, iNum + '[2] Page must have three additional portfolio images.');
                var aImageData = this.evaluate(function(sUrl) {
                    var images = __utils__.findAll('div.gallery img');
                    return images.map(function(el) {
                        return {
                            "url": sUrl,
                            "src": el.src.replace(/^http:\/\/.+?\//g, "\/"),
                            "width": el.naturalWidth,
                            "height": el.naturalHeight
                        };
                    });
                }, sUrl);
                for (i = 0; i < aImageData.length; i++) {
                    var oImg = aImageData[i];
                    if (oImg.width !== 600 || oImg.height !== 480) {
                        test.fail(iNum + '[3] Size of ' + oImg.src + ' is incorrect: ' + oImg.width + 'x' + oImg.height + 'px');
                    } else {
                        test.pass(iNum + '[3] Size of ' + oImg.src + ' is correct.');
                    }
                }
            });
        });
    });
    casper.run(function() {
        this.echo("");
        test.done();
    });
});

Note the use of __utils__.findAll() from the clientutils module. As the name implies, it retrieves all DOM elements that match a specific CSS selector (in this case, portfolio images).

Having obtained our image list, we then use map() to create the aImageData array, where each item is an object containing page URL, image source, width, and height. At this point we have all the data we need to run the image dimension tests.

Run the script above and your output will look something like this (most URLs omitted for brevity):

Final Thoughts

CasperJS is a fast, versatile solution to functional web testing, including automated regression testing and identification.

The test suite for our portfolio images included 300+ tests, with the whole batch typically finishing in under 45 seconds. Assuming it takes a manual tester 30 to 45 seconds to check one URL for regressions, automation immediately provides a whopping 200-300x speedup.

CasperJS has a well-thought-out API and excellent documentation with lots of examples to build your scripts upon.

If you are familiar with (asynchronous) JavaScript, you can get up and running with Casper in next to no time.