Selenium is an automated software testing framework for web applications. It drives a real browser automatically, as if a real person was doing the clicking and typing. We use it here at The Economist for functional / black-box / acceptance testing, the kind of tests that are repetitive and time-consuming to step through manually.

We’ve setup a system that runs our full suite of regression tests in parallel, using virtual machines in the Rackspace cloud, on every commit to trunk. The entire suite takes just under 8 minutes to run, which is the length of the longest test. A screencast of each individual test run is captured in full motion MP4 video, and made available as a build artifact in Hudson for anyone to download and watch. The video is captured at full resolution (1200 x 2000), but you can see a scaled down version below of an actual test running on Economist.com:

Each Hudson job is a set of tests, here they are running in parallel on the single worker machine, the Selenium Worker:

The PHP script that does the magic is called Selenium Zoetrope. From Wikipedia: “A Zoetrope is a device that produces an illusion of action from a rapid succession of static pictures”. Here’s what an actual Zoetrope looks like:

The process starts with Hudson detecting a commit to trunk (we use Launchpad and Bazaar), and building the Selenium master job, which in turn builds all the test set jobs in parallel. Each one of these test set jobs will connect to the Selenium Worker over SSH. Each one then runs Zoetrope, which starts instances of:

1. X virtual frame buffer (Xvfb), set to a large resolution (1200x2000). The Firefox browser runs happily in here, as if it was on a real desktop environment.
2. Selenium RC server, which launches and drives Firefox, in turn driven by PHPUnit tests that make use of the Selenium extension.
3. FFMPEG recording a screencast of the Xvfb.

And then it runs the tests in the given folder of trunk, and shuts everything down afterwards and reports results in JUnit format. We’ve configured 6 build executors on our worker, but on a powerful cloud machine you could easily increase this. A diagram of what’s happening here:

A human test instance is an entire copy of Economist.com running in the cloud, see the linked post for more information on how we set them up.

The good news is that we’ve open sourced Selenium Zoetrope, so please download and try. The usual caveats apply: this works for us, and it’s still alpha level and in no way generalised for public use, but it wouldn’t take much to make it so. I’ll be posting setup instructions shortly, but the Launchpad project with the source is up.

There’s lots that I’ll hopefully be writing about in the near future, including:

• PHPUnit / Selenium integration, details and gotchas.
• How to setup Hudson and the Zoetrope worker.
• Structure of an object oriented Selenium testing framework.
• The TestData Drupal module.
• Running Zoetrope locally
• Huge props to David Strauss, we worked together on this project and he was instrumental in solving several tricky problems. And everyone at The Economist who has worked hard on the continuous integration systems over the last few years.