JavaScript and Browser Automation

Selenium Conf 2016

See this talk online

Umar Hansa / @umaar
Want to see the GoPro version with heart rate data?

DevTools Tips

A developer tip, in the form of a gif, in your inbox each week.


Or search: 'chrome dev tips'

>120 tips posted so far! 🎊

The goal of this talk

  • Understanding how to automate browsers in JS
  • Evaluate some browser automation tooling
  • Learn some debugging tricks

Why JavaScript?

The NPM ecosystem

Over 1 billion downloads per week

Projects most ⭐starred⭐ on github are mainly JS


JavaScript has elegant syntax

Fat arrow

const odds = => v + 1)


[a, , b] = [1,2,3];
// a = 1
// b = 3


Before async/await (callback version)

browser.getTitle(function(title) {

Before async/await (promise version)

browser.getTitle().then(function(title) {

With async/await

title = await browser.getTitle()

Template strings

const selector = `.header`
findElement(`body ${selector}`)


Counting Wiki links

# In your terminal
npm i selenium-webdriver

const webdriver = require('selenium-webdriver');

// no need to memorise this 😧

const browser = new webdriver
    .withCapabilities({'browserName': 'chrome' })


const links = await browser.findElements(


node Wiki.js

# > 396

Getting to Philosophy on Wikipedia

Getting to Philosophy on Wikipedia

Clicking on the first lowercase link in the main text of a Wikipedia article, and then repeating the process for subsequent articles, usually eventually gets one to the Philosophy article.

const url = '';

function clickFirstLink() {
    const link = '#mw-content-text > p a[title]';;

if (linkText === 'Philosophy') {
    // We made it!
} else {;

A Google Search

The 'hello world' of browser automation

Selenium WebDriver

Official JS bindings


    .sendKeys('selenium conf 2016');


const selector = webdriver.By.css(

const link = webdriver

browser.wait(link, 2000).click();

console.log('Title: ', await browser.getTitle());

//Instead of:

    .then(title => console.log(title));


$ node --harmony-async-await search.js

# Title:  SeleniumConf UK 2016 - Home page

Error examples

Missing URL to browser.get()

  • ✅ The official JavaScript bindings
  • ✅ Solid documentation
  • ✅ Lots of reading material (Wiki) e.g. Design Patterns
  • ✅ Low level - power to do anything
  • ❗Promise Manager (ControlFlow) - however it can be turned off
  • ❗Awkward terminal output


The Google search demo...

var client = webdriverio.remote(options);
    .setValue('*[name="q"]','selenium conf 2016')
  • ❗Swallowing JavaScript Errors
  • ❗Having to call end()
  • ❗Docs showing older JS syntax
  • ✅ Easy to understand
  • ✅ Quick to get started with
  • ✅ 2.4k commits


it('performs a google search', function() {
    const linkSelector = '[href=""]';
    browser.type('[name="q"]', 'selenium conf 2016')
    console.log('Title is: ', browser.getPageTitle());
  • ❗Last release a year ago, Nov 2015
  • ❗Lots of code examples in coffeescript (api docs in JS)
  • ✅ Decent Terminal Output
  • ✅ Simple API
  • ✅ Automatic screenshot tracking



import { ClientFunction } from 'testcafe';

const getTitle = ClientFunction(() => document.title);

fixture `Google Search for Selenium Conf 2016`
    .page ``;

const text = 'selenium conf 2016';
const link = '[href=""]';

await t.typeText('[name="q"]', 'selenium conf 2016')
console.log('Title: ', await getTitle());

Custom Error

  • ✅ Nice terminal output
  • ✅ Elegant API, official JS syntax
  • ✅ Solid documentation, all in ES6/ES7
  • ✅ Automatic wait for element visibility
  • ❓ No built in getTitle
  • ❓ Still a bit new

Scraping speaker names from the conference website

Tips and tricks

Selenium - webdriver.js

Source code as documentation

Never Sleep

🙅 .sleep() / .pause()🙅

Replace existing sleeps with waitFor

Introduce errors for learning

  • Throw JS errors
  • Go to invalid pages
  • Manipulate hidden parts of the DOM

Use a MITM proxy to mock network requests

Proxy Use cases

  • Scraping: block JS + CSS from loading
  • Testing: serve static data for expensive server calls
  • Learning: manipulate the response, learn more HTTP

Code review everything

Encourage best practices

Your tests are still code

  • Lint
  • Code Review
  • Pair
  • Documentation

Extra considerations

  • Caching
  • User agents
  • Screen dimensions
  • Restarting the browser with a fresh profile


Split your tests up based on priority

Run p1 tests before a deploy

Run p3 tests overnight


Take time to learn async with callbacks, promises and await

async/await is still async code

Further Reading

Browser Automation Libraries

Live edit WebDriverJS code

node --inspect google-search-example.js

More details on how this works

Thank you

Twitter: @umaar for feedback or ideas