Distributed Testing with Selenium & Docker Containers (Part I)

As seen at TechCrunch Disrupt, the UnifyID Google Chrome extension plays a prominent place in the user’s first impression with the product. As such, it is critical that the experience be smooth, user-friendly, and most important, reliable. In minute 2:10 of the demo, the Amazon login fields are replaced with the UnifyID 1-click login. This requires DOM (document object model) manipulation on the web page which isn’t challenging on a single website; however, when scaling your code across sites in multiple languages and formats, finding the right login form and place to insert our logic is challenging.

How to find the right elements in here?
How to find the right elements in here and generalize well?

Parsing through the noise of variations on HTML structures, CSS naming conventions, and general web development is like the wild west–but despite this, there are ways to gain sanity, namely in testing.

As most UIs are tested, we utilize Selenium and its Javascript binding. For developers who have had some experience with Selenium, it becomes painfully obvious very early on that speed is not its forte. The testing environment must handle testing functionality across dozens, hundreds, and thousands of sites while maintaining the fast, iterative nature of development. If every test case takes an hour to finish, then our continuous integration would lose its magical properties (you know, moving fast without breaking things).

Surfing at speed through the codebase.
Surfing at speed through the codebase, knowing everything will be ok.

The solution to distribute testing across multiple computers will require the following:

  1. Must run multiple instances of the tests with the same code but on different websites.
  2. Must run all of the tests in parallel.
  3. Must collect all results.

In order to simplify my work, I discovered that Selenium has this neat feature called Selenium Grid that allows you to control testing in a remote machine so that the local machine doesn’t have to run the tests itself. You just run the grid in another machine, expose the port, and remote connect to that port by using settings found in your chromedriver.

This was amazing! This meant that we could have machines entirely dedicated to running a bunch of Selenium webdrivers (which are considerably RAM consuming) without killing our CI server. Plus, we could easily scale by just spawning more of these machines and more of these web drivers.

Finally!
Finally! Scalability!

As a result, the design evolved to this:

  1. Set up a Selenium server to run a grid of browsers. Run Chrome.
    1. Package under a Docker image for easy deployment.
    2. Run under an Amazon AWS instance for easy scaling.
  2. The CI server has to be able to send requests to the Grid.
  3. The tests have to run in parallel.
    1. Create a Javascript library to make tests into chunks that can be sent over to the server.
  4. Finally, the results have to be merged together and sent back to the CI server.

For 1, I downloaded selenium-server-standalone-2.53.1.jar from “http://selenium-release.storage.googleapis.com/index.html?path=2.53/” and saved as selenium-server-standalone.jar.

Run hub with:

java -jar selenium-server-standalone.jar\
-role hub

Run node with:

java -jar selenium-server-standalone.jar -role node\
-hub http://localhost:4444/grid/register\
-browser browserName=chrome,version=52.0,\
          chrome_binary='/usr/bin/google-chrome',\
          platform=LINUX,\
          maxInstances=5

Now to connect to the grid, we can run from JavaScript:

Look out for my next post which will delve deeper into parts 2, 3, and 4 above.

Thanks for Hacking!