Testing WYMeditor

Unit tests vs. Selenium tests

For testing, two different systems are used; Selenium and QUnit.

The vast majority of code is covered by the QUnit unit test suite.

Some functionality can’t be tested by QUnit, and thus is tested using Selenium.

Since QUnit is faster than Selenium, it is the preferred test platform. In absence of compelling reason otherwise, new tests should be written in QUnit.

All tests should pass in all supported browsers.

If ever any tests fail, please file a bug.

All of the following assumes you’ve completed the process of Configuring Your Development Environment

QUnit test suite

Running in real browsers

Start a server using

$ grunt server

After the server is running, by default, it serves on port 9000 so point a browser at http://localhost:9000/test/unit/index.html.

Note

If testing in a virtual machine, for better performance, disabling virtual memory may help.

Running in a headless browser

In addition to running the QUnit test suite in real browsers, running it from the command line in a headless PhantomJS browser is also possible.

This is how the travis-ci test suite runs.

$ grunt test

If tests fail, an error will be output.

Running with various versions of jQuery

The QUnit test suite can be run with various versions of jQuery.

Instructions for doing that are provided in the following sections.

The default version of jQuery that is used is the lowest supported version.

Testing with jQuery versions other than the default involves the automatic fetching of the desired jQuery version from Google’s CDN.

In real browsers

To do this when running tests in a browser, append the URL parameter ?jquery=<version> to the test suite URL.

For example, for jQuery version 1.7.0 use http://localhost:9000/test/unit/?jquery=1.7.0.

In a headless browser

To do this when running tests from the command line, include the parameter --jquery=<version> when running the test task.

For example, for jQuery 1.6.0 use

$ grunt test --jquery=1.6.0

Travis CI

WYMeditor is set up on Travis CI.

The QUnit test suite is run automatically using the test Grunt task with several different versions of jQuery whenever commits are submitted to the Git repository for the project.

Any submitted pull requests should pass these tests.

Selenium tests

QUnit operates within the confines of JavaScript. Some functionality can’t be tested with JavaScript, exclusively. This includes browsers’ reaction to entered input like keyboard, mouse and touch. It also includes focus–related activity like window switching. It also includes the execCommand funcionality.

For testing these areas, we rely on Selenium. It controls real browsers in ways that JavaScript can’t.

Since QUnit is faster than Selenium, it is the preferred test platform. In absence of compelling reason otherwise, new tests should be written in QUnit.

WYMeditor’s Selenium test suite is written in Python, using Python bindings for Selenium.

Running Selenium tests

  1. Install the Selenium 2 Python bindings, roughly following these instructions.

    The specific version of the python Selenium bindings and the nose testing framework we require are defined in a pip requirements file located at wym_selenium/requirements.txt. To install these, we recommend that you first create an isolated python virtualenv.

    $ mkdir -p ~/.virtualenvs
    $ virtualenv ~/.virtualenvs/wym
    
  2. Then use pip to install the requirements:

    (wym)$ cd /path/to/wymeditor
    (wym)$ pip install -r selenium_requirements.txt
    
  3. To run the Selenium tests, you’ll first need to serve the src directory with a web server. If you have Python installed, then you can simply open a terminal and run:

    $ cd /path/to/wymeditor
    $ make testserver
    

    You’ll need to keep this terminal open when running the tests.

  4. Then you can use make once again (in another terminal) to actually run the tests:

    $ source ~/.virtualenvs/wym/bin/activate
    (wym)$ cd /path/to/wymeditor
    (wym)$ make selenium