Previously on our Web Testing Series

Basics of Writing Nightwatch Web Tests

Learn how to test most scenarios on the web using these 3 techniques → finding elements, interacting, and asserting their properties.

Engineering Blog | Nightwatch.jsBharath Raja

In our previous chapter, we learnt how to test basic scenarios on the web using these three powerful techniques.

• Finding Elements →  browser.element.find()
• Interacting with Elements → .click() and .sendKeys()
• Asserting Elements → .getText().assert.contains()

Overview of this Post

Today we will learn some advanced techniques and use cases in web testing.  This will also help you understand the extensive capabilities of what Nightwatch can do. We will expose you to the following scenarios and concepts.

Concepts:

• Test Hooks
• Keyboard Shortcuts & Clipboard
• OS & Browser Info
• Executing client JS
• Actions API
• iFrame
• Async/Await
• Multi-tab Interaction
• Emulating geolocation

For the following tests, you can run them individually after writing each of them or run them all together at the end. You can also use this post as a reference to come back when you're facing similar scenarios.

Complex Scenarios & Concepts

🪝 Test Hooks

Test hooks are special functions that allow developers to perform specific actions at different stages of the test execution process. In our home.spec.js test, we always want to go to the homepage of our nightwatch website. Instead of writing browser.navigateTo('/') in every test, we can add it under a beforeEach hook that will get executed before each test is run. We can also close the browser after every test with browser.end().

describe('Nighwatch homepage', function() {
  beforeEach(browser => browser.window.maximize().navigateTo('/'))
  afterEach(browser => browser.end())
  ...
})

If you want to execute something before or after initiating the tests from a file (home.spec.js), you can do so using before and after hooks. You can also perform an operation before starting or just before exiting the test runner using Global Hooks.

📎 Keyboard Shortcuts & Clipboard

We learnt how to simulate key press using .sendKeys() in our previous post, so shortcuts, including copy and paste, should be fairly straightforward. Pressing CONTROL or COMMAND or SHIFT will hold that key until Keys.NULL is pressed, which will release all the held keys. But the clipboard isn't directly accessible for us to test. We check the clipboard's contents by pasting it on an input element using the keyboard shortcut (⌘ + v / Ctrl + v) and checking the value attribute of the input. Let's test the following scenario,

Click the copy button on the home page and verify if the text is copied.

it('Should copy the installation command on copy button click', function (browser) {
  browser.element.findByText('Copy').click()
  browser.element.find('#docsearch').click()
  const $inputEl = browser.element.find('.DocSearch-Modal .DocSearch-Form input')
  $inputEl.sendKeys([browser.Keys.COMMAND, 'v'])
  $inputEl.getAttribute('value').assert.contains('npm init nightwatch')
})

If we're running the tests on Windows or Linux, we need to replace browser.Keys.COMMAND with browser.Keys.CONTROL. This brings us to the next topic.

🌐 OS & Browser Info

We can find the details about the browser and the platform the test is running in from the browser.capabilities object. These three are the most commonly used – .platformName, .browserName and .browserVersion. We can use this for the previous example and rewrite our test to run on multiple platforms.

const is_mac = browser.capabilities.platformName.toLowerCase().includes('mac')
...
$inputEl.sendKeys([is_mac ? browser.Keys.COMMAND : browser.Keys.CONTROL, 'v'])

🤝 Executing client JS

Next, we'll try to execute some JS in the browser client. This can be done in Nightwatch using executeScript or executeAsyncScript function.

Execute a client script to increase the size of the logo and change the text of the "Get Started" button to "{Client Side Execution}". Try finding the button with the new text and click it.

it('Should should change with client script', async function (browser) {
  const change_text = "{Client Side Execution}"
  browser.executeScript(function (new_text) {
    const $hero_cta = document.querySelector('.hero__action-button--get-started')
    $hero_cta.innerHTML = new_text
    $hero_cta.style.background = '#ff7f2b'
    document.querySelector('header .navigation-list').style.display = 'none'
    document.querySelector('header .navigation__logo').style.width = '900px'
  }, [change_text])
  browser.pause(1000) // Pausing just to notice the changes
  browser.element.findByText(change_text).click()
  browser.assert.titleMatches('Getting Started')
})

✍ ️Actions API

The Actions API provides granular control over exactly what designated input devices can do. Nightwatch provides an interface for 3 kinds of input sources:

• A key input for keyboard devices
• A pointer input for a mouse, pen or touch devices
• Wheel inputs for scroll wheel devices

This can be done within the existing .perform() command. These are the available actions – .clear(), .click([element]), .contextClick([element]), .doubleClick([element]), .dragAndDrop(from, to), .insert(device, ...actions), .keyDown(key), .keyUp(key), .keyboard(), .mouse(), .move([options]), .pause(duration, ...devices), .press([button]), .release([button]), .sendKeys(...keys) and .synchronize(...devices).

Example:

browser
  .perform(function () {
    const actions = this.actions({ async: true })

    return actions
      .keyDown(Keys.SHIFT)
      .move({ origin: el })
      .press()
      .release()
      .keyUp(Keys.SHIFT)
  })

🖼 iFrame

One of the trickiest aspects of the web is iFrame. These embedded iFrames render a completely different webpage and have their own browsing context and document, allowing for a new non-inherited web page inside yours. Nightwatch has a way to switch to these documents using the .frame() method.

Enter the email and click subscribe inside an iframe.

it('Should allow for substack subscription', function (browser) {
  const iframe_selector = '.footer__wrapper-inner-social-subscribe iframe'
  browser
    .executeScript(function (iframe_selector) {
      document.querySelector(iframe_selector).scrollIntoView()
    }, [iframe_selector])
  browser.element.find(iframe_selector).setAttribute('id', 'test-nightwatch-123')
  browser.frame('test-nightwatch-123')

  browser.element.find('input[type=email]').sendKeys('test@nightwatchjs.org')
  browser.element.find('button[type=submit]').click()

  browser.ensure.alertIsPresent()
  browser.alerts.accept()
  browser.element.findByText('Sign out').assert.present()
})

We execute a client script to scroll to the bottom of the page as the iframe we want to interact with is lazy loaded. This scroll to the bottom can also be achieved using the Actions API as follows.

browser
  .perform(function() {
    return this.actions().move({
      origin: browser.element.find(iframe_selector),
    })
  })

🚦 (Not) Using Async/Await

Javascript is a language built around the asynchronous nature of User Interfaces. This was a pain for developers, and to control the order of execution of our code, we started using callbacks. But soon, we ended up in callback hell, and we hated it. Then the beauty of Promises was introduced, and soon in ES7, we got a syntax sugar for it called async/await.

When writing our tests in Javascript, one often comes across the same issue, "Why isn't this code executing" or "This line isn't supposed to execute at this time". We at Nightwatch cared a lot about solving this issue with programming in Javascript. We implemented an async command queue behind the scenes, so you, as a tester, don't have to worry about it.

Nightwatch has internal command queue, so you don't have to worry about the async issues of Javascript

But on the rare occasion that you're taking a value out of Nightwatch API and using it in a vanilla Javascript, then async and await becomes essential. While testing, this is very unlikely, but if you do face such a situation, you can use await to get the value out since every Nightwatch API returns a Promise.

Example:

it('Use values from the webpage', async function(browser) {
  const href = await browser.element.find('.navigation-list li a').getAttribute('href').value
  // You can do anything with the href value here after
  console.log(href)
  MyAPI.track(href)
})

🗂 Multi-tab Interaction

One of the scenarios we commonly come across during browsing is clicking a link that opens in a new tab. Nightwatch allows you to test this scenario by giving you an API to switch between different open documents.

Click on the GitHub icon and check the URL of the new tab opened.

it('Should lead to the GitHub repo on clicking the Github icon', async function (browser) {
  browser.element.find('ul.navigation-list.social li:nth-child(2) a').click()
  // wait until window handle for the new window is available
  browser.waitUntil(async function () {
    const windowHandles = await browser.window.getAllHandles()
    return windowHandles.length === 2
  })

  const allWindows = await browser.window.getAllHandles()
  browser.window.switchTo(allWindows[1])

  browser.assert.urlContains('github.com/nightwatchjs')
})

Every tab is considered a window by the browser. We have a different window object in each of them, after all. Here we wait for the new tab to be available, switch to the new tab, and check if the browser URL matches our GitHub repo URL.

🌍 Emulating geolocation

When your website or web app is changing based on the location from which it is being accessed, it becomes important to test your website for all these locations. With the introduction of Chrome DevTools Protocol, Nightwatch supports mocking the geolocation of the browser during the test run with just one command.

Check the address from 3 different locations on the globe.

it('sets and verifies the geolocation to Japan, USA and Denmark', function (browser) {
  const location_tests = [
    {
      location: { latitude: 35.689487, longitude: 139.691706, accuracy: 100 },
      // Tokyo Metropolitan Government Office, 都庁通り, Nishi - Shinjuku 2 - chome, Shinjuku, 163 - 8001, Japan
      test_text: 'Japan',
    },
    {
      location: { latitude: 40.730610, longitude: -73.935242, accuracy: 100 },
      // 38-20 Review Avenue, New York, NY 11101, United States of America
      test_text: 'New York',
    },
    {
      location: { latitude: 55.676098, longitude: 12.568337, accuracy: 100 },
      // unnamed road, 1550 København V, Denmark
      test_text: 'Denmark',
    }
  ]

  const waitTillLoad = async function () {
    const geo_dom_class = await browser.element.find('#geolocation_address')
      .getAttribute('class').value
    return !geo_dom_class.includes('text-muted')
  }

  location_tests.forEach(obj => {
    browser.setGeolocation(obj.location).navigateTo('https://www.where-am-i.co/')
    browser.waitUntil(waitTillLoad)
    browser.element.find('#geolocation_address').getText().assert.contains(obj.test_text)
  })
})

Explanation:

We go through each location in the location_tests array and set the browser to that geolocation. Then wait for the address to be loaded using .waitUntil(fn) by checking if the text-muted class has disappeared. Then we verify if the address has the right text.

For the test to work, you should first visit www.where-am-i.co and allow the website to access your location.

Don't worry. You could always reset the permission for the location after the testing is done.

Coming up Next

Writing scalable tests with Page Object Model

Today you learnt a lot about how to do in-depth testing of websites and web apps to simulate complex use cases and scenarios using Nightwatch. In the next chapter of our series on web testing, we will explore test writing patterns and introduce you to Page Object Model (POM). We’ll go through different patterns that improve the structure and maintainability of your test code, and you’ll learn how to implement the POM – a design pattern promoting reusability and modularity in your test scripts.

Join Our Community 💬

If you have any questions, don't hesitate to visit our Discord server and say hello. Our community is always available to provide support, share insights, and assist you with any testing-related inquiries you may have. We welcome your active participation and look forward to connecting with you in our Discord community. You can also reach out to us via Twitter.

Happy testing! 🎉

Previously on our Web Testing Series

Get Started with Web Testing Using Nightwatch

Discover how to get started with web testing using Nightwatch, a powerful no-compromise test automation framework. Learn how to install Nightwatch, set up your testing environment, and write basic end-to-end tests.

Engineering Blog | Nightwatch.jsBharath Raja

We learnt how to set up a testing environment with Nightwatch and successfully wrote our first end-to-end test. Here is a quick recap,
Install: npm init nightwatch
Write:  browser.navigateTo('/').assert.textEquals('h1', '<TEXT>')
Run: npx nightwatch test

Overview of this Post

Checking a text isn't enough of an end-to-end test for ensuring the reliability of websites and web apps. The web is made of complex elements and involves a lot of user interactions. In this tutorial, we'll teach you how to test some of the common scenarios continuing from the first test we wrote.

Writing web tests majorly surround these 3 components,

• Finding an element
• Interacting with the element
• Testing the element and its properties

We'll cover these three primary aspects of testing to be able to write tests for the following scenarios.

• Click the "Get Started" button and verify if we've gone to the installation page.
• Click on Search, type the query "frame", wait for the results, press the down arrow to the second result, and press enter. Verify if we've gone to the .parentFrame() documentation page.

You can also watch the video tutorial of this post.

Find 👀

The first step in web testing is to find DOM elements in the page we want to test. Nightwatch allows you to find elements using a variety of selectors. The most common approach is to use CSS selectors. You can do this using the .element.find() API.

Beyond that, you can search for elements on the page through various other means shown below. In the following examples, we'll use .find(), .findByText() and .findByPlaceholderText(). You can learn more about all the selectors available here.

You can also keep searching deeper inside elements by chaining all the find APIs. This really makes it easy to get the exact element you want.

Example:

browser.element.find('main').find('.hero').findByText('Get Started')

Waiting

Sometimes the element we're looking for isn't ready, and we might need to wait for it. A good example is typing in an input box and waiting for the results to be populated. You can do this using the .waitUntil(<status>) API.

Example:

// Find the element with class "DocSearch-Modal" and wait for it to be visible
browser.element.find('.DocSearch-Modal').waitUntil('visible')

Now that we've learnt how to find an element, we can move on to interacting with it.

Interact ✍️

The next step is to interact with the web. The most common scenarios are users clicking a button or filling out a form. This part of testing becomes essential when testing a dynamic web app or website.

Nightwatch helps you simulate these actions through the element Interaction API browser.element.find().<interaction_api>(). These are the available interaction APIs: clear, click, dragAndDrop, sendKeys, setAttribute, setProperty, submit, update, upload, submitForm, updateValue, uploadFile, clickAndHold, doubleClick, rightClick and moveTo.

Beyond this, you can also do complex user actions using .perform(). But for the scope of this blog, we'll only be using .click() and .sendKeys().

Example:

// Find the button "Get Started" and click it
browser.element.findByText('Get Started').click()

// Find the input box with placeholder "Search docs" and type "frame"
browser.element.findByPlaceholderText('Search docs').sendKeys('frame')

// Press the down arrow and press Enter
browser.element.findByPlaceholderText('Search docs').sendKeys([browser.Keys.ARROW_DOWN, browser.Keys.ENTER])

Now that we've learnt how to interact with an element, we can move on to verifying aspects of it.

Assert ✅

The final step is to test if what we're looking for matches what's present on the web page. This involves taking the attribute of the element we need and verifying it.

Finding the attribute

The most commonly tested aspects of an element are its text, attributes (including class), and value (for input elements). This can be achieved using .getText(), .getAttribute() and .getValue(). You can read more about all the element states here.

Testing

Once we have found the value we need, the next step is to test it. The tests can be written as loosely or as strictly as you prefer based on these three methods: equals, contains and matches.

Example:

// Check if the h1 text contains "Install Nightwatch"
browser.element.find('h1').getText().assert.contains('Install Nightwatch')

// Check if the "autocomplete" attribute of the input box is exactly "off"
browser.element.findByPlaceholderText('Filter by title').getAttribute('autocomplete').assert.equals('off')

Negation

To check if something should "not" be equal or should "not" contain, it's as simple as adding a .not after the .assert.

// Check if the h1 does not text contains "Selenium"
browser.element.find('h1').getText().assert.not.contains('Selenium')

Element Status

You can assert the state of the element using .visible(), .present(), .selected() and .enabled() after the .assert. This is very similar to the .waitUntil status we learnt earlier in the waiting section and can be used interchangeably.

// Check if the element with class "DocSearch-Dropdown-Container" is present
browser.element.find('.DocSearch-Dropdown-Container').assert.present()

Document Status

You can assert the title and URL of the document using the browser.assert.url[Contains/Matches/Equals](<string>) and browser.assert.title[Contains/Matches/Equals](<string>).

// Check if the title contains "Getting Started"
browser.assert.titleContains('Getting Started')

Putting it all together 🧩

With all the things we've learnt so far, let's write tests for these two scenarios.

Scenario: Should lead to the installation page on click of "Get Started"

Code

  it('Should lead to the installation page on click of Get Started', function (browser) {
    browser.navigateTo('/')
    browser.element.findByText('Get Started').click()
    browser.element.findByPlaceholderText('Filter by title').waitUntil('visible')
    browser.element.find('h1').getText().assert.equals('Install Nightwatch')
    browser.assert.titleEquals('Getting Started | Developer Guide | Nightwatch.js')
    browser.assert.urlContains('nightwatchjs.org/guide/quickstarts')
    browser.element.findByPlaceholderText('Filter by title')
      .getAttribute('autocomplete').assert.equals('off')
    ;
    browser.end()
  })

Explanation

First, we navigate to the home page using .navigateTo('/'). This is possible because previously, in the setup of our project, we set the base_url of our project to be the Nightwatch home page. You can edit this in the nightwatch.conf.js file. After the navigation, we find the button with the text "Get Started" and click it. For situations when the link is routed in the front end and doesn't reload the page, it is recommended to wait for the action to complete. Here we'll wait for the left-side search bar to become visible. Then we'll check the installation page by asserting the title, the URL, and the h1 element. We'll also assert the "autocomplete" attribute of the search bar to showcase attribute assertions.

Scenario: Should allow search and show correct results

Code

  it('Should allow search and show correct results', function (browser) {
    browser.navigateTo('/')
    browser.element.find('#docsearch').click()
    browser.element.find('.DocSearch-Modal').waitUntil('visible')
    
    const search_input = browser.element.findByPlaceholderText('Search docs') 
    search_input.sendKeys('frame')
    browser.element.find('.DocSearch-Dropdown-Container').assert.present()
    search_input.sendKeys([browser.Keys.ARROW_DOWN, browser.Keys.ENTER])

    browser.element.find('h1').getText().assert.contains('.frameParent')
    browser.end()
  })

Explanation

We'll navigate to the home page, find the search icon and click on it. We'll wait for the modal to open and be visible. Then we'll type in the text "frame" in the search input bar. We wait for the results to load by checking the presence of ".DocSearch-Dropdown-Container", and once the results are loaded, we'll press the down arrow and enter. This will lead to the docs of the second result .frameParent(), and we verify the h1 text to confirm that.

Final File

The final home.spec.js should look like this.

describe('Nighwatch homepage', function () {
    
  it('Should have the correct title', function(browser) {
    browser
      .navigateTo('/')
      .assert.textEquals('h1', 'Introducing Nightwatch v3')
      .end()
    ;
  })
    
  it('Should lead to the installation page on click of Get Started', function (browser) {
    browser.navigateTo('/')
    browser.element.findByText('Get Started').click()
    browser.element.findByPlaceholderText('Filter by title').waitUntil('visible')
    browser.element.find('h1').getText().assert.equals('Install Nightwatch')
    browser.assert.titleEquals('Getting Started | Developer Guide | Nightwatch.js')
    browser.assert.urlContains('nightwatchjs.org/guide/quickstarts')
    browser.element.findByPlaceholderText('Filter by title')
      .getAttribute('autocomplete').assert.equals('off')
    ;
    browser.end()
  })

  it('Should allow search and show correct results', function (browser) {
    browser.navigateTo('/')
    browser.element.find('#docsearch').click()
    browser.element.find('.DocSearch-Modal').waitUntil('visible')
    
    const search_input = browser.element.findByPlaceholderText('Search docs') 
    search_input.sendKeys('frame')
    browser.element.find('.DocSearch-Dropdown-Container').assert.present()
    search_input.sendKeys([browser.Keys.ARROW_DOWN, browser.Keys.ENTER])

    browser.element.find('h1').getText().assert.contains('.frameParent')
    browser.end()
  })

})

Let's run the tests with the same command we learnt in the previous post.

npx nightwatch test

After the browser is done with the tests, the output should look like this.

Coming up Next

Advanced Techniques and Scenarios in Web Testing

Today you learnt how to test the web for the common use cases using these three techniques → Finding, Interacting and Asserting. In the next chapter of our series on web testing, we will learn about advanced techniques in web testing, such as test hooks, multi-tab interactions, iFrames, copy/paste, using async/await, executing client JS, emulating geolocation, handling shadow DOM, actions API and more. Stay tuned for our upcoming post on Advanced Techniques and Scenarios in Web Testing.

Join Our Community 💬

If you have any questions, don't hesitate to visit our Discord server and say hello. Our community is always available to provide support, share insights, and assist you with any testing-related inquiries you may have. We welcome your active participation and look forward to connecting with you in our Discord community. You can also reach out to us via Twitter.

Happy testing! 🎉

Video Tutorial

A Series on Web Testing with Nightwatch

Welcome to our blog series on testing websites and web apps in real browsers with Nightwatch. We will go through learning about testing step by step.

• Getting started (you’re here)
• Complex scenarios and tests for websites
• Test writing patterns focussing on Page Object Model (POM)
• Generating HTML reports with DOM history for thorough analysis
• Creating custom commands using Nightwatch

Additionally, we'll explore two powerful features of Nightwatch:

• The Inspector tool, which enables you to write tests using a graphical interface, and
• The Chrome Recorder, which allows you to create tests without writing code. We'll guide you through the process of using these tools effectively.

To ensure comprehensive testing, we'll also discuss

• Running tests on CI – GitHub Actions
• Cross-browser testing using BrowserStack
• Running tests on mobile browsers

By the end of this series, you will have a solid foundation in web testing with Nightwatch, enabling you to ensure the quality and reliability of your web projects.

In this initial blog post, we will focus on getting started with Nightwatch. From setting up Nightwatch to writing your first test and generating reports, we'll provide a step-by-step guide to get you started on your web testing journey using Nightwatch. You can also watch the video tutorial of this post.

A Brief on the Testing Space

Software testing encompasses various types of testing to ensure the quality and reliability of applications. Unit testing validates individual code units, integration testing checks component interactions, and API testing focuses on API functionality. Visual regression testing detects visual changes, while end-to-end testing covers the entire application flow. Accessibility testing ensures inclusive user experiences. Cross-browser testing guarantees compatibility across different browsers, and mobile testing ensures optimal performance on mobile devices. Each type of testing serves a specific purpose and contributes to ensuring a high-quality software product.

A Brief on Nightwatch 🦉

Nightwatch is a no-compromise test automation framework that simplifies web and mobile testing for developers. Nightwatch comes with a powerful set of tools to write, run and debug your tests across web and native mobile applications. It seamlessly integrates with popular testing tools and services, enabling efficient test execution and analysis of results.

Setting up Nightwatch

Let's get started. First, we need to set up Nightwatch in our development environment. Whether you're starting a new project or integrating Nightwatch into an existing one, we've got you covered.

npm init nightwatch <project-name>

If you already have an existing web application, go into the root folder and run the following.

npm init nightwatch

Nightwatch will then ask a series of questions. Please answer the following.

? Select testing type to setup for your project 
    ❯◉ End-to-End testing
? Select language + test runner variant 
    ❯ JavaScript / default
? Select target browsers 
    ❯◉ Chrome
? Enter source folder where test files are stored 
    test
? Enter the base_url of the project 
    https://nightwatchjs.org/
? Select where to run Nightwatch tests On 
    ❯ localhost
? Allow Nightwatch to collect completely anonymous usage metrics? 
    ❯ Yes
? Setup testing on Mobile devices as well? 
    ❯ No, skip for now

It is as simple as that to set up Nightwatch! Once you answer the questions, Nightwatch installs all the necessary dependencies and prepares your development environment for writing tests. For both new projects and existing ones, it's the exact same journey. You can read more about installation in our Guide here.

Your First Test

Now comes the exciting part – writing your first test! We'll guide you through the fundamentals of creating test scripts with Nightwatch.

Nightwatch supports the popular BDD interfaces for writing tests - describe and it format. Let's start by defining the spec for our first test. In this test, we will go to the homage page of Nightwatch and check if the title is correct → “Introducing Nightwatch v3”.

Create a file home.spec.js under /test folder with the following code.

describe('Nighwatch homepage', function() {
  it('Should have the correct title', function(browser) {
    browser.navigateTo('/').assert.textEquals('h1', 'Introducing Nightwatch v3')
  })
})

Explanation

browser.navigateTo('/') - We request the browser to go to the path '/' from the launch URL we've defined during installation. You can edit this in the config - nightwatch.conf.js

.assert.textEquals('h1', 'Introducing Nightwatch v3') - We verify if the text in the element with the selector h1 is equal to "Introducing Nightwatch v3" using the textEquals function.

Running

Once your test script is saved, it's time to execute it with Nightwatch. Go to your command line and enter the following command.

npx nightwatch test

You should see the Chrome browser open up, navigate to the Nightwatch home page and close. The command line should have the following.

Reports and Debugging

After executing tests, Nightwatch generates detailed reports that provide invaluable insights. Copy the HTML report file path and paste it into a browser with the prefix file://, and you'll see the following report. We can check this report for debugging our tests as we write complex tests and face issues.

Coming up Next

Exploring Complex Testing Use Cases and Advanced Techniques

You've learnt how to set up a testing environment with Nightwatch and successfully write an end-to-end test. In our next blog post, we will dive deeper into the realm of website testing – strategies for handling intricate testing scenarios, such as multi-tab interactions, complex selectors, iFrames, copy/paste, attribute assertions, using async/await, interacting with DOM elements, executing client JS and more. Stay tuned for our upcoming blog post on "Complex Testing Scenarios for Websites".

Join Our Community 💬

If you have any questions, don't hesitate to visit our Discord server and say hello. Our community is always available to provide support, share insights, and assist you with any testing-related inquiries you may have. We welcome your active participation and look forward to connecting with you in our Discord community.

Happy testing! 🎉

Video Tutorial

We are excited to announce Nightwatch v3, a no-compromise test automation framework to write, run and debug your tests across web and native mobile applications.

The 3 Pillars of v3

Nightwatch v3 has been redesigned around the three core pillars of developer experience, mobile-first, and one framework for everything testing.

Developer experience

The entire experience for installing, writing, running, and debugging tests has been redesigned for an amazing developer experience:

Installing – Get started in 60 seconds

The new installation helper in Nightwatch makes the setup of the framework and dependencies a breeze. Answer a few questions, and Nightwatch will do the entire setup for you based on your answers.

Writing – Auto-wait. Selector playground. Simplified commands.

Nightwatch automatically waits for elements to appear before attempting to interact with them, reducing the need for explicit wait commands in the test code. You can create and test selectors instantly with the Selector Playground and write test scripts that are easy to maintain.

Running – IDE integration. Default parallelization. Improved performance.

Run your Nightwatch tests from VS Code itself without switching screens to run or debug your tests. Nightwatch also supports running tests in parallel on your local machine or a cloud grid. We’ve also improved our performance significantly for up to 25% faster tests than Nightwatch v2.

Debugging – DOM History. Pause and Debug. HTML reporter.

View DOM changes and relive the tests visually as they happened with DOM History. Using Nightwatch Inspector, you can stop execution at any point in the test, get selector recommendations and debug issues. You can easily identify any issues in your Nightwatch tests using the built-in HTML reporter with test statuses, assertion logs, and raw HTTP logs.

Mobile first

Native Apps, Mobile Web, and Flexible Mobile Testing.

Test your native mobile and mobile web applications along with the flexibility to run tests on your own mobile device, emulators, or cloud platforms. Nightwatch also supports Safari, both desktop and mobile, out of the box. The best part is that you can configure all of this easily with the mobile-helper.

One framework for everything testing

Test everything with a single framework.

Avoid switching test automation frameworks for different types of testing. Here is a list of all the tests that you can run with Nightwatch:

• Unit testing
• Integration testing
• API testing
• Component testing
• Visual Regression testing (VRT)
• Accessibility testing (A11Y)
• End-to-end testing (E2E)
• Mobile testing

Faster than ever

This update significantly increases the performance of Nightwatch. Nightwatch v3 is up to 2.7x faster than Cypress when testing on multiple VMs. Here’s a detailed comparison of Nightwatch against Cypress.

Try it yourself

What’s next?

Here’s a quick preview of what’s coming next.

Even better Developer Experience

• New commands for easy installation of more testing types & plugins
• Video recording and HTML reporter improvements
• Keeping the drivers in sync with browsers using Selenium manager
• Online Playground - write tests without having to set up Nightwatch
• Switch between Nightwatch VRT and Percy with just a config change

Integrations Support

• Svelte end-to-end and component testing with Nightwatch
• Nightwatch integration with BrowserStack Test Observability
• Improve CucumberJS reporter and output

Component testing in Nightwatch has been refined with version 2.4 and support for testing React components (via the @nightwatch/react plugin) has been significantly improved. We also released a new plugin for using the popular Testing Library with Nightwatch – @nightwatch/testing-library, available since Nightwatch v2.6.

We're now going to build a detailed example of how to use Nightwatch and Testing Library to test React components. We'll be Complex example available on the React Testing Library docs, which is written with Jest.

This tutorial we'll cover how to:

1. set up a new React project with Vite, which is also what Nightwatch uses internally for component testing;
2. install and configure Nightwatch and Testing Library;
3. mock API requests with the @nightwatch/api-testing plugin;
4. write a complex React component test using Nightwatch and Testing Library.

Step 0. Create a new project

To get started, we'll create a new project with Vite:

npm init vite@latest

Select React and JavaScript when prompted. This will create a new project with React and JavaScript.

Step 1. Install Nightwatch and Testing Library

Testing Library for React can be installed with the @testing-library/react package:

npm i @testing-library/react --save-dev

To install Nightwatch, run the init command:

npm init nightwatch@latest

Select Component testing and React when prompted. This will install nightwatch and the @nightwatch/react plugin. Choose a browser to install the driver for. We'll be using Chrome in this example.

1.1. Install @nightwatch/testing-library plugin

Since v2.6, Nightwatch provides its own plugin for using the Testing Library queries directly as commands. We're going to need it to write our test later on so let's install it now:

npm i @nightwatch/testing-library --save-dev

1.2 Install @nightwatch/apitesting plugin

The example contains a mock server that is needed to test the component. We'll be using the integrated mock server that comes with the @nightwatch/apitesting plugin. Install it with:

npm i @nightwatch/apitesting --save-dev

Step 2. Create the Login component

We'll use the same component as in the React Testing Library docs. Create a new file src/Login.jsx and add the following code:

// login.jsx
import * as React from 'react'

function Login() {
  const [state, setState] = React.useReducer((s, a) => ({...s, ...a}), {
    resolved: false,
    loading: false,
    error: null,
  })

  function handleSubmit(event) {
    event.preventDefault()
    const {usernameInput, passwordInput} = event.target.elements

    setState({loading: true, resolved: false, error: null})

    window
      .fetch('http://localhost:3000/api/login', {
        method: 'POST',
        headers: {'Content-Type': 'application/json'},
        body: JSON.stringify({
          username: usernameInput.value,
          password: passwordInput.value,
        }),
      })
      .then(r => r.json().then(data => (r.ok ? data : Promise.reject(data))))
      .then(
        user => {
          setState({loading: false, resolved: true, error: null})
          window.localStorage.setItem('token', user.token)
        },
        error => {
          setState({loading: false, resolved: false, error: error.message})
        },
      )
  }

  return (
    <div>
      <form onSubmit={handleSubmit}>
        <div>
          <label htmlFor="usernameInput">Username</label>
          <input id="usernameInput" />
        </div>
        <div>
          <label htmlFor="passwordInput">Password</label>
          <input id="passwordInput" type="password" />
        </div>
        <button type="submit">Submit{state.loading ? '...' : null}</button>
      </form>
      {state.error ? <div role="alert">{state.error}</div> : null}
      {state.resolved ? (
        <div role="alert">Congrats! You're signed in!</div>
      ) : null}
    </div>
  )
}

export default Login

Step 3. Create the component test

One of the founding principles of Testing Library is that tests should resemble how users interact with the application as much as possible. When writing component tests in Nightwatch using JSX, we need to write the test as a component story using the Component Story Format, a declarative format introduced by Storybook.

This enables us to write tests that focus on how the component is used, rather than how it's implemented, which is in line with the Testing Library philosophy. You can read more about this in the Nightwatch docs.

The great thing about using this format to write our tests is that we can use the same code to write stories for our components, which can be used to document and showcase them in Storybook.

3.1 Login with valid credentials test

Create a new file src/Login.spec.jsx and add the following code, which does the same as the complex example written with Jest:

To render the component using JSX in Nightwatch, we simply create an export for the rendered component, optionally with a set of props. The play and test functions are used to interact with the component and verify the results.

• play is used to interact with the component. It's executed in the browser context, so we can use the screen object from Testing Library to query the DOM and fire events;
• test is used to verify the results. It's executed in the Node.js context, so we can use the Nightwatch browser object to query the DOM and verify the results.

// login.spec.jsx
import {render, fireEvent, screen} from '@testing-library/react'
import Login from '../src/login'

export default {
  title: 'Login',
  component: Login
}

export const LoginWithValidCredentials = () => <Login />;
LoginWithValidCredentials.play = async ({canvasElement}) => {
  //fill out the form
};

LoginWithValidCredentials.test = async (browser) => {
  // verify the results
};

Add the mock server

The example uses a mock server to simulate a login request. We'll be using the integrated mock server that comes with the @nightwatch/apitesting plugin.

For this we'll use the setup and teardown hooks which we can write directly in the test file. Both hooks are executed in the Node.js context.

We also need to set login endpoint to http://localhost:3000/api/login in the Login component, which is the url to the mock server.

Complete test file

The complete test file will look like this:

// login.spec.jsx
import {render, fireEvent, screen} from '@testing-library/react'
import Login from '../src/Login'

let server;
const token = 'fake_user_token';
let serverResponse = {
  status: 200,
  body: {token}
};

export default {
  title: 'Login',
  component: Login,
  setup: async ({mockserver}) => {
    server = await mockserver.create();
    server.setup((app) => {
      app.post('/api/login', function (req, res) {
        res.status(serverResponse.status).json(serverResponse.body);
      });
    });

    await server.start(mockServerPort);
  },

  teardown: async (browser) => {
    await browser.execute(function() {
      window.localStorage.removeItem('token')  
    });
    
    await server.close();
  }
}

export const LoginWithValidCredentials = () => <Login />;
LoginWithValidCredentials.play = async ({canvasElement}) => {
  //fill out the form
  fireEvent.change(screen.getByLabelText(/username/i), {
    target: {value: 'chuck'},
  });

  fireEvent.change(screen.getByLabelText(/password/i), {
    target: {value: 'norris'},
  });

  fireEvent.click(screen.getByText(/submit/i))
};

LoginWithValidCredentials.test = async (browser) => {
  const alert = await browser.getByRole('alert')
  await expect(alert).text.to.match(/congrats/i)

  const localStorage = await browser.execute(function() {
    return window.localStorage.getItem('token');
  });

  await expect(localStorage).to.equal(fakeUserResponse.token)
};

Debugging

One of the main benefits of using Nightwatch for component testing, besides having the same API available for end-to-end testing, is that we can run the tests in a real browser, instead of a virtual DOM environment, such as JSDOM.

This allows us to use the Chrome Dev Tools to debug the tests.

For example, let's go ahead and add a debugger statement in the LoginWithValidCredentials.play function:

LoginWithValidCredentials.play = async ({canvasElement}) => {
  //fill out the form
  fireEvent.change(screen.getByLabelText(/username/i), {
    target: {value: 'chuck'},
  });

  fireEvent.change(screen.getByLabelText(/password/i), {
    target: {value: 'norris'},
  });
  
  debugger;
  
  fireEvent.click(screen.getByText(/submit/i))
};

Now, let's run the test with --debug and --devtools flags:

npx nightwatch test/login.spec.jsx --debug --devtools

This will open a new Chrome window with the Dev Tools open. We can now set a breakpoint in the Dev Tools and step through the code.

3.2 Login with server exception test

The original example from the Testing Library docs also includes a test for the case when the server throws an exception.

Let's try to write the same in Nightwatch. This time we'll use just the test function, since we can interact with the component this way as well. As we've mentioned earlier, the test function is executed in the Node.js context, and it receives the Nightwatch browser object as argument.

We'll also need to update the mock server response to return a 500 status code and an error message. This we can easily accomplish by writing a preRender test hook on the LoginWithServerException component story.

export const LoginWithServerException = () => <Login />;
LoginWithServerException.preRender = async (browser) => {
  serverResponse = {
    status: 500,
    body: {message: 'Internal server error'}
  };
};

LoginWithServerException.test = async (browser) => {
  const username = await browser.getByLabelText(/username/i);
  await username.sendKeys('chuck');

  const password = await browser.getByLabelText(/password/i);
  await password.sendKeys('norris');

  const submit = await browser.getByText(/submit/i);
  await submit.click();

  const alert = await browser.getByRole('alert');
  await expect(alert).text.to.match(/internal server error/i);

  const localStorage = await browser.execute(function() {
    return window.localStorage.getItem('token');
  });

  await expect(localStorage).to.equal(token)
};

4. Run the test

Finally, let's run the test. This will run the LoginWithValidCredentials and LoginWithServerException component stories in Chrome.

npx nightwatch test/login.spec.jsx

To run the test without opening the browser, we can pass the --headless flag.

If all goes well, you should see the following output:

[Login] Test Suite
────────────────────────────────────
ℹ Connected to ChromeDriver on port 9515 (1134ms).
  Using: chrome (108.0.5359.124) on MAC OS X.

Mock server listening on port 3000

  Running <LoginWithValidCredentials> component:
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[browser] [vite] connecting...
[browser] [vite] connected.
  ✔ Expected element <LoginWithValidCredentials> to be visible (15ms)
  ✔ Expected element <DIV[id='app'] > DIV > DIV> text to match: "/congrats/i" (14ms)
  ✔ Expected 'fake_user_token'  to equal('fake_user_token'): 

  ✨ PASSED. 3 assertions. (1.495s)

  Running <LoginWithServerException> component:
──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
[browser] [vite] connecting...
[browser] [vite] connected.
  ✔ Expected element <LoginWithServerException> to be visible (8ms)
  ✔ Expected element <DIV[id='app'] > DIV > DIV> text to match: "/internal server error/i" (8ms)
  ✔ Expected 'fake_user_token'  to equal('fake_user_token'): 

  ✨ PASSED. 3 assertions. (1.267s)

  ✨ PASSED. 6 total assertions (4.673s)

5. Conclusion

That's it! You can find the complete code for this example in the GitHub repository. PRs are welcome.

Fell free to drop by the Nightwatch Discord if you have any questions or feedback.

Storybook integration is part of Nightwatch starting with v2.4 via the @nightwatch/storybook plugin, currently available for React.

Storybook is arguably the most popular component library around at the moment, with a steady user base for React and growing communities for Vue, Svelte, Angular, and other UI libraries. It’s being used by thousands of teams around the world to build UIs with reusable components.

The popularity of Storybook can probably be attributed to identifying one essential need that frontend developers have today: in order to be able to build modern complex UIs, frontend teams need to be able to design, build, and test UI components in isolation. Storybook calls this technique component driven development (CDD).

Component Driven Development

Storybook aims to deliver a complete solution for managing large component libraries by offering compelling tools for documenting and testing. Each component is rendered in isolation with its various states and requirements.

Each representation of a component with various properties and arguments is called a story and all the stories belonging to a particular component are collected in a .stories.js[x] (or .ts[x]) file. Stories are written in a declarative syntax and each story simulates a particular component variation, by rendering the component with a specific set of props and mock data.

Here’s a basic component story example:

// Histogram.stories.js|jsx

import React from 'react';

import { Histogram } from './Histogram';

export default {
  title: 'Histogram',
  component: Histogram,
};

const Template = (args) => <Histogram {...args} />;

export const Default = Template.bind({});
Default.args = {
  dataType: 'latency',
  showHistogramLabels: true,
  histogramAccentColor: '#1EA7FD',
  label: 'Latency distribution',
};

Storybook is distributed as a Node web application which loads the existing stories and displays them in a sophisticated management interface which has integrated tools for documenting and testing the components.

The Storybook app can run locally or it can be deployed on a web server. In fact many popular component libraries are available to the public.

How to test component stories

Presently, Storybook is used mainly as a great collaboration tool between frontend developers and designers to document the UI components and even entire pages.

That is all fine until testing comes into the picture and starts complicating everything. That’s when the real fun begins, because in order to effectively develop UI components, an efficient way to easily test them is also needed.

Automated testing of components is needed to ensure that all stories are rendered properly with the specified props and mock data. Furthermore, more complex components can have stories which require simulating user actions and verifying behaviour.

Out of the box, Storybook provides built-in support for several testing strategies of components:

• visual tests
• interactive tests
• accessibility tests
• snapshot testing

So there is some built-in support for testing and also a CLI test runner which uses Playwright and Jest under the hood. However its strength still remains more on the design side of the whole web application development spectrum.

In order to have strong confidence over the code quality for the components that you build, you will need to either extend the stories with additional support for component testing or import stories into other tests.

Integrate Nightwatch into your Storybook

Nightwatch can assist here with filling the gap between UI component design+development and QA+testing. In order to add support for automated testing of components in a Storybook environment, you only need to add install Nightwatch and extend your existing stories with test functionality.

In the following section we will attempt to do just that. We’ll take an existing Storybook which is available publicly on Github and we’ll go through these steps:

1. install and configure Nightwatch
2. run the existing component stories with Nightwatch
3. extend one of the complex stories and with Nightwatch testing functionality

Let us begin then. We’ll use the World Food Programme’s Ui kit available on Github at https://github.com/wfp/designsystem.

Install Dependencies

We’ll first have to fork the project. My own fork is located at github.com/pineviewlabs/wfp-designsystem. I’m going to clone the project locally and install the existing dependencies:

The --legacy-peer-deps flag is needed to make some older dependencies install. If you get npm audit messages, you can disregard them for now, since this is just a tutorial.

Run Storybook

Once everything is installed, you can try running Storybook locally. To do that, run:

npm run storybook

This will build the Storybook files and run the project on your local host.

At the moment of writing this guide, the Storybook version used in the WFP design kit is 6.2. You can try to upgrade it to at least v6.5 but I already tried that and got a bunch of errors, so for the moment I am sticking with the 6.2 just to get everything working together.

Install Nightwatch

Nightwatch can be installed with just one command:

npm init nightwatch@latest

This will prompt you to install the browsers and location of the test files. I have selected all of them Chrome, Firefox, Safari, and Edge. Edge needs to be installed separately but the init tool will generate the config needed in Nightwatch.

For the source folder location enter the following:

src/components/**/*.stories.js

For base url enter:

http://localhost:9000/

We’ll also need to install the Storybook plugin for Nightwatch:

npm install @nightwatch/storybook

You might need to pass --legacy-peer-deps again, if you’re on newer NPM versions.

Last step is to load and configure the plugin in Nightwatch. To do that, edit nightwatch.conf.js and add the following right after src_folders:

// nightwatch.conf.js
module.exports = {
  // .. other settings
  plugins: ['@nightwatch/storybook'],

  '@nightwatch/storybook': {
    start_storybook: false, 
    storybook_url: 'http://localhost:9000/'
  }
}

Nightwatch can automatically start/stop Storybook for us during the test run, but since this particular version of Storybook takes a bit of time to load, we have kept start_storybook to false and we will run it manually.

Run stories in Nightwatch

The @nightwatch/storybook plugin is designed to work in such a way that component tests can be added on top of existing component stories, instead of expecting users to import stories in tests. When using Nightwatch with Storybook, the test is the .stories file itself.

In our case, we are going to run the existing src/component/**/*.stories.js files as tests. But before we can do that, we need to configure one last thing.

Customise esbuild

Nightwatch uses esbuild under the hood in order to parse JSX files and this project contains some syntax features which esbuild doesn’t support them by default. However, we can support them by enabling compilation using babel.

To do so, edit nightwatch.conf.js and add the following, right after the line which contains '@nightwatch/storybook':

// nightwatch.conf.js
module.exports = {
  // other settings here...
  esbuild: {
    babel: {
      filter: /\.[cm]?js$/
    }
  }
}

Add Babel

Add a babel.config.json file with the following content so that esbuild can find it:

{
  "exclude": ["node_modules/**"],
  "presets": [
    "@babel/preset-env",
    "@babel/preset-react"
  ],
  "plugins": [
    "@babel/plugin-proposal-export-namespace-from",
    "@babel/plugin-proposal-export-default-from"
  ]
}

Run Nightwatch

Now the time has come to actually run the component stories. The project already has some unit-level type of component tests written in Jest and using Enzyme. We’re going to not touch them and instead work with the actual .stories files.

We’re just going to do an exploratory test run using Chrome, and we’ll do it in --serial mode so we can observe the browser:

npx nightwatch --env chrome --serial

Running all the stories in the project in serial mode could take a while. But we can try to run them in parallel (you can replace “chrome” with either “firefox”, “safari”, or “edge”, depending on which browsers you have installed).

By default it will pick up the number of test workers based on the number of CPU cores, but we will set it to 4. We’ll also run it in --headless mode so the browsers won’t popup and distract us (note that headless mode is not available in Safari):

npx nightwatch --env chrome --workers=4 --headless

So by now hopefully we have a working Nightwatch installation in the WFP Storybook project. At the moment, the stories will only be rendered and Nightwatch will perform a visibility check to ensure that everything is OK.

Next step though is to extend one of the stories and add some Nightwatch specific functionality.

Extend stories with Nightwatch test

We’re going to pick one of the larger stories in the project and start adding additional test functionality. It seems like src/components/Form/Form.stories.js it’s a good candidate in terms of complexity so we’re going with that one.

Let’s run it first separately:

npx nightwatch src/components/Form/Form.stories.js -e chrome

There are four different stories in the file and the output should look similar to this:

[Form.stories.js component] Test Suite
───────────────────────────────────────────────────────────────────────
  Using: chrome (107.0.5304.110) on MAC OS X.


  Running "Default" story:
───────────────────────────────────────────────────────────────────────
  ✔ Passed [ok]: "components-forms-form--default.Default" story was rendered successfully.

  ✨ PASSED. 1 assertions. (1.078s)

  Running "Detailed Form" story:
───────────────────────────────────────────────────────────────────────
  ✔ Passed [ok]: "components-forms-form--detailed-form.DetailedForm" story was rendered successfully.

  ✨ PASSED. 1 assertions. (583ms)

  Running "Login" story:
───────────────────────────────────────────────────────────────────────
  ✔ Passed [ok]: "components-forms-form--login.Login" story was rendered successfully.

  ✨ PASSED. 1 assertions. (411ms)

  Running "Contact" story:
───────────────────────────────────────────────────────────────────────
  ✔ Passed [ok]: "components-forms-form--contact.Contact" story was rendered successfully.

  ✨ PASSED. 1 assertions. (513ms)

  ✨ PASSED. 4 total assertions (5.768s)

Add Nightwatch file uploading test

Nightwatch can run Storybook interaction tests and accessibility tests, together with supporting the hooks api. However, for this article we're only going to explore the test() function which Nightwatch adds on top of the Component Story Format. You can read more about the Storybook integration on the Nightwatch documentation.

We’re going to extend the DetailedForm story so let’s go ahead and edit the src/components/Form/Form.stories.jsand add the following right before that line with export const Login (around line 400):

The test() function receives both the Nightwatch browser API (which can be used to issue commands and run assertions) and a component object which points to the root element in the story and it is compatible with Nightwatch commands and assertions.

In this example will simulate the uploading of a file, using Nightwatch uploadFile() api command and we’ll verify if the file drop zone element has been updated.

// src/components/Form/Form.stories.js
DetailedForm.test = async (browser, {component}) => {
  await browser.uploadFile('input[type="file"]', require.resolve('./README.mdx'));

  const fileElementContainer = await browser.findElement(`.${settings.prefix}--file-container`);
  const elementHasDescendants = await browser.hasDescendants(fileElementContainer);
  
  await browser.strictEqual(
    elementHasDescendants, true, 'The file dropzone element has been populated.'
  );
};

Also add this import at the top of the file:

// src/components/Form/Form.stories.js
import settings from '../../globals/js/settings';

We can now run only the DetailedForm story:

npx nightwatch src/components/Form/Form.stories.js -e chrome --story=DetailedForm

Hopefully all is well and the output should look like below (there might be some warnings on top, but those aren’t critical):

[Form.stories.js component] Test Suite
───────────────────────────────────────────────────────────────────────
  Using: chrome (107.0.5304.110) on MAC OS X.


  Running "Detailed Form" story:
───────────────────────────────────────────────────────────────────────
  ✔ Passed [ok]: "components-forms-form--detailed-form.DetailedForm" story was rendered successfully.
  ✔ Passed [strictEqual]: The file dropzone element has been populated.

  ✨ PASSED. 2 assertions. (1.217s)

Conclusion

That’s pretty much it for now. I might write another similar post using a different public storybook library. Or maybe you might want to contribute one? The great humans at Storybook have recently started to put together a collection of public storybook libraries in their Component Encyclopedia so there’s plenty of opportunity to experiment and maybe even make open-source contributions.

If you made it thus far in the tutorial, I salute you! You are now ready to make a contribution to an open-source project that’s being used at the United Nations, which is quite something.

Thanks for reading and don’t forget to wear your hat as you please – indoors or out.

We are excited to share that you can now run and debug your Nightwatch tests from VS Code itself, without having to switch screens or windows. The extension gets auto-activated if you have a Nightwatch conf.js file. You can also add it manually with the Install Nightwatch action.

As developers, we often have moments where we are in a flow or a zone and the quality of code we produce at that moment is the best of us. However, the smallest of distractions could knock you out of that zone. The new Nightwatch extension for VS Code allows you to stay focussed by helping you to run and debug your tests from VS Code itself.

Here’s what you can do with the extension:

Easy Installation

The extension will be auto-activated if you have a nightwatch.conf.js file. You can also add it manually with the Install Nightwatch action. The installation guide will ask you for your choice of browsers to be installed.

Run and Debug your tests

Run single or multiple tests from your sidebar or with the Run command. Easily debug your tests by setting up a breakpoint. VS Code will pause your tests at the breakpoints you have set.

View tests results

This extension simplifies test results for you. You can view individual or aggregate test results and can also view the output of the test execution in the VS code terminal.

For more details, check out the VS Code extension here. You can also raise issues on GitHub or discuss any questions you have on Discord.

We are excited to introduce the new Nightwatch Chrome Recorder extension that helps you generate Nightwatch test scripts without writing a single line of code.

Test without writing a single line of code

Nightwatch Chrome Recorder helps you to export Chrome DevTools recording as Nightwatch test scripts. This will speed up your development cycle by creating integrated or end-to-end tests instantly. In fact, a recent survey by Gitlab revealed that 49% of organizations had already started using Low Code/No code tools.

This is based on the Chrome DevTools Recorder, a tool that allows you to record and replay test actions directly in the browser. We decided to use it since it’s plugged into the browser, giving the convenience of not switching context or dealing with other third-party tools.

How to use

Here is how you can generate your Nightwatch script in 4 simple steps:

1. Install the extension
2. Navigate to the Recorder section in Chrome DevTools
3. Start recording and perform all the actions in your test like a typical user would
4. Stop the recording. You can now click on export and select ‘Export as Nightwatch Test Script’ to get your tests.

Please refer to our documentation if you need more details, or reach out to us on Discord if you have any questions.

Nightwatch v2.2 brings its own integrated HTML reporter which better aggregates test results along with other details. In this blog, we will set up the Nightwatch project which automatically publishes the generated HTML report on GitHub pages after the test tun.

Here is the preview of Nightwatch HTML Report:

Create Project

1. Setup Nightwatch

I am using the nightwatch-examples repo as a template to set up some example tests.

You can also use the Nightwatch init command which would manage all the config and dependencies as well as add some example tests in your project.

npm init nightwatch

2. Run the tests locally

To check if the tests are running locally,in the nightwatch-examples template there is already a test script in package.json:

// package.json

"scripts": {
    "test": "nightwatch --env chrome"
  },

Run the tests with:

npm test

Define GitHub workflow

Let's move on to automating the execution and publishing of the test report with GitHub actions.

1. Create Test Workflow

Create a directory .github/workflows and add test.yml

//test.yml

name: nightwatch.tests
on:
  push:
    branches: 
    - main
jobs:
  tests: 
    name: 'nightwatch tests'
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@master
      - uses: actions/setup-node@v2
        with:
          node-version: "14.x"
      - name: Install dependencies
        run: npm ci
      - name: Run Nightwatch tests
        continue-on-error: true
        run: xvfb-run --auto-servernum npm test -- --env chrome
      - name: "Publish test results"
        uses: peaceiris/actions-gh-pages@v3.7.3
        with: 
          github_token: ${{ secrets.GITHUB_TOKEN }}
          publish_dir: './tests_output/nightwatch-html-report'
          keep_files: true
          user_name: <user-name>
          user_email: <user-email>

I have selected the main branch to run GitHub Actions. In the following steps, it downloads the dependencies, runs tests, and publishes reports to a branch gh-pages which will be used to host the HTML page.

Some configuration details:

• continue-on-error is set to true as we need to publish reports in case of failure.
• publish_dir points to the directory where the Nightwatch HTML report is generated by default. These can be changed using the Nightwatch output configuration.
• user_name, user_email, and github_token are used to create a commit and push content to the selected branch.

Running tests in headless mode is not always 100% reliable, so we have added  support for using xvfb to run tests in Chrome.

2. Setup GitHub Pages

And the final step is to enable GitHub Pages for your project under Settings → Pages.

And we are good to go ⭐.

To verify everything is set up correctly push all the changes to GitHub which triggers Actions.

3. Triggering Test Workflows

Typically we want to run the jobs not only when tests are updated but also when there is a new deployment or a need to schedule jobs.

Conclusion

Using GitHub Actions helps us to automate our test execution and using GitHub pages we can publish our test reports. The report contains all the test details including raw HTTP logs which would help in debugging later on.

With just one command, set up everything to run Nightwatch tests in under 60 seconds. Our new onboarding client aims to provide a powerful onboarding experience to the developers setting up Nightwatch for the first time or in their existing project. Learn how to quickly set up the config, and automatically install the webdrivers and the required packages.

Running it

It all starts with the command npm init nightwatch

> npm init nightwatch
Need to install the following packages:
  create-nightwatch
Ok to proceed? (y)

This is an npm alias for using npx create-nightwatch . The npx is a command for executing package binaries without downloading to the local dependency.  npm then installs the package in a cache and runs the binary. If you are starting a new project, you can also run npm init nightwatch <directory> .

The Flow of Choices

The setup is based on a list of questions to accommodate a variety of use cases.

Test Runner

Nightwatch also supports other test runners. You can also pick Mocha or Cucumber as a test runner apart from Nightwatch.

JavaScript / Typescript

Nightwatch supports typescript for test files after v1.6.0. So you can choose to have the test setup in Javascript or Typescript.

===============================
Nightwatch Configuration Wizard
===============================

Just answer a few questions to get started with Nightwatch:

We'll setup everything for you :-)

? What is your Language - Test Runner setup? (Use arrow keys)
❯ JavaScript - Nightwatch Test Runner
  JavaScript - Mocha Test Runner
  JavaScript - CucumberJS Test Runner
  TypeScript - Nightwatch Test Runner
  TypeScript - Mocha Test Runner

Local / Remote

You can setup the config and downloads based on if you're going to be running the test locally or on a remote machine or both.

 Where do you want to run your e2e tests? (Use arrow keys)
❯ On my local machine
  On a remote machine (cloud)
  Both

Browser Selection

You can pick the browsers you'll be testing on, and the config will be created for them. We provide a multi-selection option so you can pick as many browsers you want to test on. You can also use the selenium-server when testing on the local machine.

(Local) Where you'll be testing on? (Press <space> to select, <a> to toggle all, <i> to invert selection, and <enter> to proceed)
❯◉ Firefox
 ◯ Chrome
 ◯ Edge
 ◯ Safari
 ◯ IE (requires selenium-server)
 ◯ Local selenium-server

If you are running from a Mac, safaridriver is present by default but must be enabled.

? Enable safaridriver (requires sudo password)? (Use arrow keys)
  Yes
❯ No, I'll do that later.

For remote testing, you should provide the host address and port of the remote machine.

? (Remote) What is the host address of your remote machine? (localhost)
? (Remote) What is the port on which your test backend is running on your remote machine? (80)

Other parts

Apart from the fundamental differences in the ways you test, we also keep adding other extra configs you can add to the questionnaire. As of May '22, the list is as follows.

• Directory location for the end-to-end tests
• the base_url of your project
• Using the --generate-config to create only the config and not download any dependencies

Result

If --generate-config is not provided, the script installs the necessary packages.

Installing the following packages:
- nightwatch
- @nightwatch/selenium-server
- @cucumber/cucumber

For local tests, the script also installs the wedrivers based on your browser selection.

Installing/Updating the following webdrivers:
- geckodriver
- chromedriver
- iedriver
- safaridriver

By default, the configuration file nightwatch.conf.js is generated and examples at tests/nightwatch-examples . If the nightwatch.conf.js is already present then you can choose to overwrite the existing config file or to create a new one.

? Do you want to overwrite the existing config file? No, create a new one!
? What should your new config file be called? ie-windows-test.conf.js

Then you can run the examples with this command.

npx nightwatch ./tests/nightwatch-examples

We Love Open Source - BrowserStack

BrowserStack offers free web and mobile testing for open source projects. Test your web and mobile apps on our Real Device Cloud. Apply with project details to qualify

We have sponsored 1500+ open source projects

A brief introduction

Behaviour-driven Development (BDD)

Behaviour-driven Development is a software development technique that has evolved from TDD (Test-driven Development), an approach or programming practice where the developers write new code only when the automated test case fails. BDD encourages teams to use concrete examples to formalize a shared understanding of how the application should behave between various tech and non-tech teams. Tests are more user-focused and based on the system's behaviour. In BDD, “Given-When-Then” is the proposed approach for writing test cases.

Cucumber.js and Nightwatch

Cucumber is an Open Source framework that supports BDD using human language syntax and uses Gherkin as the parser. Cucumber is implemented in most mainstream programming languages, including JS. Nightwatch recently introduced the new integrated test runner for Cucumber.js in the 2.0 alpha release. No other plugins are necessary except the Cucumber library itself with this new update. In this post, we'll write and run some BDD tests to verify the search results of a museum page using Cucumber.js and Nightwatch.

Getting Started

1. Install Nightwatch, Cucumber and the driver for the browser you want to test. Ensure that the browser driver version is the same as your local browser.

npm i @cucumber/cucumber nightwatch chromedriver --save-dev

2. Set Cucumber as the test_runner in the nightwatch.conf file.

{
  test_runner: {
    type: 'cucumber',
    options: {
      feature_path: '*/*.feature',
      ...
    }
  src_folders: ['features/step_definitions'],
  ...
}

You can read more about configuring Cucumber with Nightwatch.

Writing the BDD tests

Cucumber uses the Gherkin syntax to create human-readable tests. Gherkin uses a set of special keywords to give structure and meaning to executable specifications. Let's create a features/rijksmuseum.feature and write some tests to search inside the Rijksmuseum website.

Feature: Rijksmuseum Search
Background: Goto website
  Given I open the Rijksmuseum page
  And I dismiss the cookie dialog
  Then the title is "Rijksmuseum Amsterdam, home of the Dutch masters"

We'll write some Background that gets executed before every scenario. In this case, we open the museum website, close the popup, and verify the page's title.

Scenario 1

@search @nightwatch
Scenario: Searching for Night watch
  Given I search "night watch"
  Then Body contains "Operation Night Watch"
  Then Body contains "The Night Watch, Rembrandt van Rijn, 1642"

We have two tags here, namely search and nightwatch so that we can run tests based on tags. This scenario searches for “night watch” on the museum search page and verifies the search results contain “Operation Night Watch".

Scenario 2

@search @cucumber
Scenario: Searching for cucumber
  Given I search "cucumber"
  Then Body contains "Muskusroos (Rosa moschata) en komkommer (Cucumis sativus)"

These scenarios are very clear and use simple human language. The next step is to define what each of these steps means to the automated tests.

Using Nightwatch behind the scenes

Inside the features/step_definitions/rijksmuseum.js file, let's write definitions for the steps. Let's start with opening the museum page.

const {Given, Then, When} = require('@cucumber/cucumber')

Given('I open the Rijksmuseum page', function() {
  return browser.navigateTo('https://www.rijksmuseum.nl/en')
});

As you notice, we have access to the Nightwatch's browser object inside our definitions file. This access is enabled because of the new 2.0 update. We also set the src_folders: ['features/step_definitions'] in the config. Cucumber provides a way to write Cucumber Expressions, an alternative to Regular Expressions, with a more intuitive syntax.

When('I search {string}', function(searchTerm) {
  browser.click('a[aria-label="Search"]')

  return browser.waitForElementVisible('#rijksmuseum-app')
    .setValue('input.search-bar-input[type=text]', [searchTerm, browser.Keys.ENTER])
})

Nightwatch has a built-in extendable assert/verify library with the same methods, which perform assertions on elements.

Then('the title is {string}', function(title) {
  return browser.assert.titleEquals(title)
})

Running the tests

Nightwatch automatically detects and configures the test runner based on the installed driver. Since we're manually adding the config, let's set the default test_setting in the nightwatch.conf file. You can read more about the default config. The following snippet shows a simple setting for running it in chrome. For more information about default settings, you can read the test environments docs and the config docs.

  test_settings: {
    default: {
      desiredCapabilities: { browserName: 'chrome' },
      webdriver: { start_process: true },
    },
  },

Let's run the Nightwatch tests.

npx nightwatch

You can add more parallels to the test with --parallel 2 flag. Alternatively, you can run specific tests using the cucumber tags flag --tags @search. You can check the configuration from the Nightwatch examples repo and the Developer Guide.

Conclusion

We've covered the basics of BDD testing and how to write your first test using Cucumber and Nightwatch, thanks to the new integrated test runner in the 2.0 release. So update your package now and start using Cucumber.

We Love Open Source - BrowserStack

BrowserStack offers free web and mobile testing for open source projects. Test your web and mobile apps on our Real Device Cloud. Apply with project details to qualify

We have sponsored 1500+ open source projects

Last year in November we've began working on adding component testing to Nightwatch and now I am delighted to present the first version which is ready for public use. So far, it supports Vue and React components but our plans is to support all kinds of web components in the near future.

Component testing in Nightwatch is the next logical step since we cover the browser integration and we have built-in assertion libraries and reporters. We only needed a fast and reliable way of rendering the componets in isolation and for that we have chosen Vite – the new frontend build tool created by the team behind the Vue.js project.

Below I will attempt to explain what component testing actually is and how Nightwatch (v2.0) tackles the various problems that arise with testing web components.

Component Testing Overview

Unit testing single page front-end applications usually presents an entire set of issues to deal with over regular approaches of unit testing. In order to reliably test a web component in isolation, we need to be able to render it first.

When testing web components, the crucial aspect is how the component is rendered. In many cases, rendering using a real browser might seem like an overkill and something clumsy that will slow everything down considerably. For instance, a large project might have a dedicated QA team who takes care of the end-to-end testing and in such a case, it might seem unnecessary to write component tests using real browsers, especially if it’s overly complicated or slow.

Rendering Using JSDom

JSDom is tool which offers the possibility to render a web component using a Node.js virtual renderer, without the need of a real browser. Everything happens in the CLI and it’s usually pretty fast.

Here’s how it normally looks like:

Rendering using Karma TestRunner/Puppeteer

The other popular approach is to use something like the Karma Runner, which is a frontend testing tool created about 10 years ago at Google. There are also variations on this theme where Puppetter and browser-based Mocha is used.

In this approach all files needed are loaded on the test renderer page where it all happens: the component is loaded together with dependencies and any testing tools needed, then the component is rendered and the test is executed in the same browser context. It looks a bit like this:

Rendering in Nightwatch

Either one of the above approaches are fine in terms of running tests and debugging, however both of them come with their one limitations, which are easy to spot:

Using JSDom rendering the limitation is clear: no real browser. However, advantages to consider are speed and access to OS-level APIs, which would make tasks like loading files or generating advanced reports more straightforward.

On the other hand, when using the Karma Runner or similar approach the advantage is clear: everything is happening in the browser and thus the test is more reliable. However, the disadvantage is that working with external files is not straightforward and also reporting is limited. I also found configuration to be quite difficult, as normally you have to combine several tools together in the setup just to get it working.

What Nightwatch aims to provide is a combination of both approaches by extending its cross-browser CLI test runner and built-in assertion library and provide an integrated and easy to use solution. Nightwatch already has support for end-to-end testing in all major browsers and comes with built-in reporting and support for parallelism out of the box.

Here’s what Nightwatch does to run a component test:

• the CLI test runner launches a real browser and navigates to a basic HTML page (the test renderer)
• inside the test renderer, it injects the Vue or React test utils and then mounts the component which needs to be tested, optionally specifying a list of plugins–in case of Vue, such as a store or a router– or props in case of React
• once the component has been successfully rendered a reference to the DOM element will be sent back to the Nightwatch CLI runner
• the CLI runner continues running the test in the same way it does for end-to-end testing; the assertions are run in the Node.js context

What is Vite?

Vite is an extremely fast build tool for modern web applications, initially created for Vue.js apps but now with support for React and other UI frameworks as well. Vite is the French word for fast, which is appropriate because among the available front-end build tools, Vite is the fastest and also one of the easiest build tools to use.

If you have used tools like Babel or Webpack you may be familiar with the problems that arise from the complexity of the build setup and the slow startup times. Vite has somehow managed to eliminate all these issues by providing a tool that it's already configured out of the box and which leverages the new capabilities of modern browsers to handle ES Modules directly, so there's no need of using tools like Babel.

In addition, Vite is using ESBuild under the hood for bundling the Javascript code and related assets, which appears to be the fastest among the bunch by a great deal.

How Does it Work?

At the moment, component testing in Nightwatch is available via our new vite-plugin-nightwatch plugin in projects that are using Vite. Our initial aim is to support React and Vue components so we can gather some initial feedback from the community.

Once Vue and React support is firmly in place, we will expand to add other frameworks like Svelte and Lit.

Installation

For now, the new Vite plugin can be installed from NPM with:

npm install vite-plugin-nightwatch

Configuration

Update your Vite configuration:

import { defineConfig } from 'vite'
import nightwatchPlugin from 'vite-plugin-nightwatch'

export default defineConfig({
  plugins: [
    // ... other plugins, such as vue() or react()
    nightwatchPlugin()
  ]
})

Update your Nightwatch config and add the plugin to the list:

module.exports = {
  plugins: ['vite-plugin-nightwatch'],
  // ... other nightwatch settings
}

For more usage details, examples, and API docs, head over to the Github page:

GitHub - nightwatchjs/vite-plugin-nightwatch: Component testing plugin that integrates Vite with Nightwatch.js. Supports Vue and React components.

Component testing plugin that integrates Vite with Nightwatch.js. Supports Vue and React components. - GitHub - nightwatchjs/vite-plugin-nightwatch: Component testing plugin that integrates Vite wi...

GitHubnightwatchjs

Complete Examples

We've also put together example projects for both React and Vue which show how the plugin can be used.

• Sample Vue ToDo app
• Sample React ToDo app

Both of these projects are under constant development so make to keep an eye on them for updates.

Vue Component Example Test

Here's a basic example of a Vue component test. It's testing a basic Form component written in Vue which is available here, as part of the plugin:

describe('Render Vue Component test', function() {
  let formComponent;

  before(async function() {
    formComponent = await browser.mountVueComponent('/test/components/vue/Form.vue', {});
  });

  it('checks the vue component', function(browser) {
    browser.expect.element(formComponent).to.be.present;
    browser.setValue('#movie-input', 'Fargo');

    const inputEl = formComponent.find('input[type="radio"][value="3"]');

    browser.expect(inputEl).to.be.present;

    browser.click(inputEl);

    browser.expect(formComponent.property('rating')).to.equal('3');
    browser.expect(formComponent.property('title')).to.be.a('string')
        .and.equal('Fargo');
  });

});

You can run this as a any other Nightwatch test or just clone/fork the project and run the included tests:

git clone git@github.com:nightwatchjs/vite-plugin-nightwatch.git
npm install
npx nightwatch test/specs/vue/testVueComponent.js --env vue

Make sure the Vite dev server is up and running, with:

npm run dev

Running the Vite dev-server programmatically from Nightwatch

It is also possible to start the Vite dev server from the Nightwatch global before hook and close it in the after hook.

Our plugin does just that in order to run its own tests  and details are provided in the Readme. Future Nightwatch plugin releases will provide this functionality out of the box and you will be able run component tests wheather or not your project is based on Vite.

Debugging Component Tests

Debugging component tests in Nightwatch isn't as straightforward as debugging a regular Node.js application or service, since Nightwatch needs to inject the code to render to component into the browser.

However, for when running the tests in Chrome, you can use the DevTools to do debugging directly in the browser. For this purpose, Nightwatch provide 2 CLI flags:

• --devtools - when this is on, the Chrome DevTools will open automatically
• --debug - this will cause the test execution to pause right after the component is rendered

Advantages of using Nightwatch for Component Testing

The main advantage and motivation for using Nightwatch for component testing is ease of use and consistency with how end-to-end testing is done. With Nightwatch you have everything installed so there is nothing more to configure in terms of testing.

And since the testing is done by the CLI test runner, we have also access to OS-level APIs and we also have access to the Vite runner APIs, so we could do more advanced integration directly between Vite and Nightwatch, but we’ll leave that one for a future update.

The only disadvantage would be that it’s going to be a bit slower than JSDom renderer and possibly also the Karma Runner. However, I believe it compensates by offering a more reliable and easy to use solution, better reporting and overall better experience. Once you enable parallelism, then I think the speed will not be a problem any longer.

I am happy to announce that Nightwatch.js is now officially a part of BrowserStack. As an open-source project, this means that its development is now a lot more secure, and its future is in excellent hands. BrowserStack has proven itself as the leading cloud infrastructure provider for both live and automated testing on desktop and mobile devices.

It also means that now Nightwatch is maintained by a full dev team and is one of the main responsibilities of the Open-source Program Office that BrowserStack has assembled recently. I have also joined this wonderful group where I’ll be working together with other Selenium and Nightwatch contributors. Our vision stays the same: to ensure that Nightwatch is the go-to solution for all cross-browser automated testing needs.

At the same time, BrowserStack is fully committed to support the entire Selenium ecosystem and so Nightwatch will still continue to work well with all Selenium-based cloud services. As for the day-to-day operations, we will be continuing to develop Nightwatch as an autonomous open-source project, in the same manner as for the past 8 years, but with the support of the world’s leading software testing platform.

Our mission remains the same – to enable software and test engineers to test their applications as quickly and as reliable as possible. I’ve never really believed in the promise of configuration possibilities. I don’t think that anyone really enjoys configuring tools and installing plugins, especially when they need to write automated tests.

That’s why Nightwatch aims to bring to the table a highly integrated solution, already packed with the right set of supporting libraries that work without any additional config needed, and written in the language that all web browsers know best – JavaScript. And since we’ve always favoured collaboration instead of competition, Nightwatch has been designed to make use of more than a decade long of experience from the Selenium project and to implement the industry standard W3C WebDriver API. Doing it any differently not only seems impractical, but it just doesn’t make sense to us.

No matter the sailboat, sailing is all about the wind they say. Maintaining open-source software is like that in many ways. Even though I don’t know the first thing about sailing, the wind is a free natural resource and available to everyone to use. From the beginning, Nightwatch has got either good or moderate wind conditions. Now it has received a strong boost and I’m very excited to see it continuing its journey at a much better speed and sailing conditions.

We're delighted to announce that BrowserStack is hosting a Nightwatch focused webinar on December 14th. Selenium committer David Burns will be experimenting with a few of the new features in Nightwatch 2 and so this is a great opportunity to check out the new version.

David heads the Open-source Program Office at BrowserStack and is one of the maintainers of the Selenium project as well as being co-editor of the W3C Webdriver spec.

Here's what's on the agenda:

• introduction new the new User Actions API from Selenium and how to use them in Nightwatch
• overview of the WebDriver Bidirectional (BiDi) APIs
• working with shadowRoot elements in Nightwatch
• creating a plugin using the new plugin API
• Q&A session

Watch the Recording

The recording is available at the link below. You do not want to miss out on this!

browserstack.com/webinars/nightwatch-2-in-action

Developing a good web application means it is usable, fully functional, and compatible. Rigorous end-to-end testing ensures rolling out a stable web application. Testing across all possible browsers guarantees a great user experience.

Nightwatch is an end-to-end automated testing solution for web applications. Developers and test engineers can write tests using Nightwatch and effortlessly target various browsers without worrying about test flakiness.

Consider an infrastructure setup with different browsers on different operation system machines. Now, the tests and infrastructure are ready. But how do we delegate the tests to run on the infrastructure? Well, Selenium Grid seamlessly stitches the two together. Selenium Grid helps in efficiently finding the accurate environment to run the tests on.

What is Selenium Grid?

Selenium Grid is a central point that aids in distributed testing, scaling multiple environments, and load-balancing incoming tests. Grid improves testing efficiency by supporting parallel testing across multiple browser-os combinations. All tests point to the Grid, and it intelligently routes them to run on the underlying infrastructure.

If you have a local or a cloud device farm, Selenium Grid will handle all the delegation needs for web testing.

Selenium Grid's functionality is divided into various components as follows:

Router

As the names suggest, the Router's primary responsibility is to route the request to the correct component. Any request to the Selenium Grid first goes to the Router. Based on the request, the Router identifies the Grid component that can handle it.

Distributor

The Distributor maintains the model of all the registered Nodes. The Distributor matches the new session request to the appropriate Node and initiates session creation. It also regularly pings the Node's health check and tracks Node's heartbeat.

Node

The Node is present on the machine that hosts the operating system and browser(s). Hence, a Node interacts with the web drivers and forwards browser commands to them. Each Node consists of a set of capabilities. A capability is a combination of browser name, browser version, and the operating system.

Upon startup, the Node registers itself with the Grid. Distributor handles the Node registration.

Session Map

The Session map contains the mapping of the session-id and the Node where the session is running. For every request for an existing session, the Router uses the Session Map to look up the Node and forwards the request to the respective Node.

New Session Queue

The New Session Queue enqueues every new session request. The New Session Queue is a FIFO queue. Meanwhile, the Distributor regularly checks if any of the Nodes have the capacity for the new session. If so, the Distributor blocks the matching slot in the Node and removes the request from the New Session Queue. The Node creates the session and responds to the client.

New Session Queue also has request retry and request timeout mechanisms.

Event Bus

Grid components leverage the Event bus to interact with each other using messages. Grid components communicate with each other via the Event Bus.

Setting up Selenium Grid

Prequisites

1. Ensure java is downloaded. If not, download and set up java on the machine that will run the Grid.
2. Download the latest jar from the Selenium downloads page.
3. Ensure that the web driver is on the system path. Refer installing browser drivers for more details. The server will auto-detect drivers on the path. This behavior is configurable.

The Selenium Grid can be set up in any of the three modes: Standalone, Hub and Node, and Fully Distributed.

In all three modes, the default server address is http://localhost:4444.

Standalone

The Standalone mode has all the Grid components in one. It is the quickest way to get started with the Grid.

java -jar selenium-server-${grid-version}>.jar standalone

Hub and Node

Hub consists of Router, Distributor, New Session Queue, Session Map, and Event Bus. The Node consists of the Node and the Event Bus to allow communication with the Hub.

Start the Hub

java -jar selenium-server-${grid-version}.jar hub

Start the Node

java -jar selenium-server-${grid-version}.jar node

Fully Distributed

Each Grid component runs independently in the fully distributed mode. All the Grid components rely on the Event Bus to communicate with each other.

Start the Event-Bus

java -jar selenium-server-${grid-version}.jar event-bus

Start the Session Map

java -jar selenium-server-${grid-version}.jar sessions

Start the New Session Queue

java -jar selenium-server-${grid-version}.jar sessionqueue

Start the Distributor

java -jar selenium-server-${grid-version}.jar distributor --sessions http://localhost:5556 --sessionqueue http://localhost:5559 --bind-bus false

Start the Router

java -jar selenium-server-${grid-version}.jar router --sessions http://localhost:5556 --distributor http://localhost:5553 --sessionqueue http://localhost:5559

Start the Node

java -jar selenium-server-${grid-version}.jar node

To check if the Grid is up, ping http://<grid-url>/status endpoint.

Each component has configurable CLI options.

Docker

Selenium Grid also supports Docker. Refer Selenium Docker to get started.

Grid UI

Navigate to http://<grid-url>/ui.

Grid UI displays the Grid model with all the Nodes. A separate tab for sessions displays the ongoing sessions and sessions waiting in the queue.

Running Nightwatch tests on Selenium Grid

Nightwatch comes packaged with a few example tests which you could use to quickly be up and running with your Grid.

For this guide, we will be assuming that Nightwatch 2.0+ is used. You can install it by running:

npm i nightwatch

Configure Nightwatch to use the Grid

The test runner requires a configuration file. The file can have the extension ".json" or ".js". The ".js" is preferred since it has more configuration options and capabilities.

Nightwatch does the heavy-lifting of generating a config file with ".js" extension if it is not present. It contains a good set of options to test with any environment. For simplicity, just the sharing the configuration file with selenium-server related options. Just ensure the the selenium-server host and port match the Grid url and you are good to go.

{
  webdriver: {},

  test_settings: {
    selenium_server: {
      selenium: {
        start_process: false,
        host: 'localhost',
        port: 4444
      }
    },

    'selenium.chrome': {
      extends: 'selenium_server',
      desiredCapabilities: {
        browserName: 'chrome',
        chromeOptions : {
          w3c: true
        }
      }
    },

    'selenium.firefox': {
      extends: 'selenium_server',
      desiredCapabilities: {
        browserName: 'firefox'
      }
    }
  }
};

Running in Parallel using Test Workers

By default the test_workers options is disabled. Enable if multiple test files should be ran parallel.

To run in parallel mode, ensure Grid's max-session CLI option is set to more than 1. By default, it is based on the number of available processors on the machine.

Update the Nightwatch config with setting the desired number of test workers:

{
  test_workers : {
    enabled: true,
    workers: 4
  }
}

Refer to the Configuration section to understand and use the configuration options.

Run Nightwatch against the Selenium Grid

To run Nightwatch against the Selenium Grid using Chrome

npx nightwatch examples/tests --env selenium.chrome

Run the tests on multiple browsers by passing a comma-separated list of environments:

npx nightwatch examples/tests --env selenium.chrome,selenium.firefox

Further Reading

• Selenium Grid Documentation