End-to-end testing Angular web apps with Protractor

You’ve completed your Angular web app and have released it for the world to consume. Congrats! But now comes the hard part: as you gain web traffic, customers are bound to suggest features to make your web app stand out. How are you going to do that without introducing new bugs or breaking existing functionality? Sure, you could release the new feature and cross your fingers that nothing goes wrong (read: bad idea). Alternatively, you could manually test the feature itself locally to ensure everything still works as expected. But that takes an unwieldy amount of time that will only grow exponentially as your web app becomes increasingly complex with new features. So what are you going to do?

The way we at Qualtrics have decided to tackle the issue is by utilizing Protractor, the nifty end-to-end testing framework made to work with Angular. Widely used for Angular 1 apps and currently in early support stages for Angular 2, Protractor interacts with your web app just like a user would by manipulating the DOM elements on the page. This includes clicking, entering text, and refreshing the page to name a few interactions. Protractor even supports Angular element selection. Did you specify a value on the $scope and don’t want to look for the HTML tag? No problem, just specify the ng-model variable and Protractor will find the HTML tag for you. The best part is that it integrates seamlessly with the $scope.$digest() cycle, which means there’s no need to explicitly sleep the testing thread waiting for execution to complete. Just kick off the thread and Protractor will wait the appropriate time for you.

In this blog post, we would like to share some lessons learned from integrating Protractor into our development cycle that help prevent breaking changes from affecting customers.

1. Using IDs on essential DOM elements for easy selector manipulation

Given the below example app, we will walk through the steps to add regression tests. We will also explain some of the optimizations that have been useful to ensure features are working as expected.

As you can see, we have a simple input box that when clicked, will change the result text and apply a style to it. So to test this code, we begin by finding the button that we need to click to trigger the action. While not required, we also split the codebase by the CSS selector code and the actual test logic.

CSS selector code

module.exports = function(options) {
    var self = _.defaults(options || {}, {});
    // add selectors to object representing web app
    self.addSelectors({
        button: by.css('button'),
        result: by.css('.output')
    });
    /**
     * @return {promise} resolve to button text
     */
    self.getButtonText = function() {
        return element(self.selectors.button).getText();
    };
    /**
     * Clicks on button to trigger action
     */
    self.clickButton = function() {
        element(self.selectors.button).click();
    };
    /**
     *  @return {promise} resolve to result text
     */
    self.getResultText = function() {
        return element(self.selectors.result).getText();
    };
    /**
     * @return {promise} resolve to true if result text is underlined
     */
    self.getResultUnderlined = function() {
        return element(self.selectors.result).getCssValue('text-decoration')
            .then(function(decoration) {
                return decoration === 'underline';
            });
    };
    return self;
};

Test Logic

var webApp = require(‘cssSelectors.js’);
describe('Test button triggering result text change', function() {
    it('should have the correct text in button', function() {
        expect(webApp.getButtonText()).to.eventually.equal('Click me!', 'Error: expected title to be "Click me"');
    });
    it('should have the correct result text before clicking button', function() {
        expect(webApp.getResultText()).to.eventually.equal('something');
        expect(webApp.getResultUnderlined()).to.eventually.equal(false);
    });
    it('should have the correct result text and style after clicking button', function() {
        webApp.clickButton();
        // because Angular waits for previous call to complete with the $scope.$digest() cycle I don’t need to explicitly chain a callback here
        expect(webApp.getResultText()).to.eventually.equal('nothing');
        expect(webApp.getResultUnderlined()).to.eventually.equal(true);
    });
});

As you can see, by splitting the CSS selection and actions performed on them, we can easily see the test cases that are performed. First, we determine that the text within the button is satisfactory, meaning that the Angular app is working as expected with the ng-value binding. Second, we test the interaction of the button on the result text. We recommend doing a before and after test for such cases since you may have unexpected behavior if the starting point is not what the test case expects.

Note that everything tested and wrapped with Protractor calls result in a returned promise. This is because of JavaScript’s asynchronous nature. We don’t want to block the operation even though Protractor may not have found the selector yet. So in the following test logic, we use the `eventually` keyword to indicate that the test suite needs to wait until the selector has been found and processed before comparing the expected value to the actual result.

For those testing against Firefox, note that simply selecting by the .output div tag won’t be sufficient to find the element text. Protractor uses jQuery selectors under the covers, so a good way to figure out what CSS selectors you’re interested in when writing the test is to manually play around in the console (under Developer Tools in browser). But back to the point, in Chrome the console is a bit smarter by allowing the $(‘output’).text() selection to find the information you’re searching for within an embedded tag. Repeating the same selection with Firefox, however, you will need to be more specific and dig down into the nested tag in order to find the text. In this case, the solution is relatively simple: we change the jQuery selection to $(‘.output span’).text() as there is only a single span selector within the div.

Extrapolating this solution to more nested spans within the same div and you can quickly see how adding those IDs will help enormously.

2. Utilizing the Protractor’s native ability to read Angular variables to avoid unnecessary element location

Following another example, let’s explore how to avoid finding the exact DOM element by utilizing the fact that Protractor can read variables directly off of the scope. In this code snippet, we have a text box that given a certain value will append it to a list in the div below.

As in the first example, there is a lack of IDs on the HTML tags and because there are now with two `input` tags, we’ll need to take advantage of a couple other selectors and action manipulators that Protractor provides to easily check the output.

CSS selector code

module.exports = function() {
    // can manually specify entire web app selectors
    var self = {
        selectors: {
            inputBox: by.model('user.firstName'),
            submitButton: by.partialLinkText('Login'),
            outputList: by.repeater('login in logins'),
        }         
    };
    /**
     * Updates input box content with specified new content
     * @param {string} newContent
     */
    self.updateInputBoxContent = function(newContent) {
        element(self.selectors.inputBox).click();
        element(self.selectors.inputBox).clear();
        element(self.selectors.inputBox).sendKeys(newContent);
    };
    /**
     * Clicks on button to trigger action
     */
    self.clickSubmit = function() {
        element(self.selectors.submitButton).click();
    };
    /**
     * @return {promise} resolve to list of elements in output list
     */
    self.outputList = function() {
        return element.all(self.selectors.outputList);
    };
    return self;
};

Test Logic

var webApp = require(‘cssSelectors.js’);
describe('Test Input Box', function() {
    it('should start with correct default state', function() {
        // can manually specify the element manipulation within the test logic, but this is not recommended
        expect(element(webApp.selectors.inputBox).getText()).to.eventually.equal('Foo');
        expect(webApp.outputList().count()).to.eventually.equal(0);
    });
    it('should output to list when button is clicked', function() {
        webApp.clickSubmit();
        // can always explicitly specify resolution handler to avoid using `eventually` keyword in `expect` statements
        webApp.outputList().then(function(list) {
            expect(list.length).to.equal(1);
            // list of elements that you can run promises against returned
            expect(list.get(0).getText()).to.eventually.equal('Foo was logged in.');
    });
    it('should end with expected state after interaction', function() {
        webApp.updateInputBoxContent('Baz');
        webApp.clickSubmit();
        webApp.outputList.then(function(list) {
            expect(list.length).to.equal(2);
            // should not modify already outputted text
            expect(list.get(0).getText()).to.eventually.equal('Foo was logged in.');
            expect(list.get(1).getText()).to.eventually.equal('Baz was logged in.');
    });
});

Using selectors, we can utilize Protractor’s ability to see elements bound to the $scope. With the first selector, we find the input box simply by specifying the ng-model that is being used within that input tag. And since the value for the submit button is now fixed (unlike in the previous example), we utilize the by.partialLinkText() function to find the DOM element directly via the outputted text. Note here that Protractor supports a similar by.linkText() function that essentially provides the same functionality, but we have found the partialLinkText() function to be superior in finding the elements due to leading/trailing whitespace that may be out of the developer’s control.

Now on the more complex action: updating the input. As seen in the updateInputBoxContent function, we split the action into 3 parts:

• click
• clear
• input

Although you may not think of it at such a low level, this is actually what users are doing when interacting the web app. As you may notice, the wrapped functions that are used in the test logic expose the high-level abstraction while covering the nitty-gritty details that Protractor needs to replicate it.

The last selector uses the ng-repeat reader that Protractor supports by specifying the same text written in the ng-repeat on the HTML tag. In the supporting function used in the test logic, we see that dealing with lists ends up becoming a bit clunky as each element in the list is itself an element. The way we’ve illustrated is only one way that the result can be tested. Another equivalent option would be to abstract out the selection of the index into the cssSelectors.js file and remove the manual selection altogether as shown below.

self.getOutputListIndexText = function(index) {
    return element.all(self.selectors.outputList).get(index).getText();
};

3. Avoiding browser.sleep() by following the $scope.digest() cycle

Last but not least is Protractor’s automatic waiting functionality. Most of the times when we have a call to an external source on the webpage, we’re likely to create the call with a timeout so that if the external source never returns, the UI won’t hang indefinitely. Consider the code snippet below that utilizes the $timeout() functionality.

(function(module) {
    'use strict';
    app.service('update', [
        '$timeout',
        function($timeout) {
            var WAIT_TIME = 10; // in ms
            return {
                startUpdate: function() {
                    $timeout(update, WAIT_TIME);
                },
                stopUpdate: function() {
                    $timeout.cancel(WAIT_TIME);
                }
            };
        }
    ]);
})(angular.module('example'));

However, as we discovered, the $timeout() function call actually halts the $scope.$digest() cycle execution. Which in turn affects the test cycle so that there is a race condition between the update return and test logic. This can be mitigated by using browser.sleep() to ensure that WAIT_TIME is exceeded before checking, or you can modify your app to take advantage of a similar but non-blocking $interval() function call. By converting both calls from $timeout to $interval, you can achieve the same result without affecting the test cycle.

$timeout(update, WAIT_TIME); => $interval(update, WAIT_TIME);
$timeout.cancel(WAIT_TIME);  => $interval(WAIT_TIME);

Summary

As shown in the examples above, modifying your Angular web app to be more test-friendly with Protractor is relatively painless from a developer’s point of view. Fully automated end-to-end testing means that any features you push out will be regression tested with a push of a button! So what are you waiting for? Get out there and test your Angular app to keep customers happy for years to come.