Automation Testing with Playwright

Why do we choose Playwright?

Playwright is a free open-source framework for automation tests. It is new to the market but is building up popularity fast. 1st release happened on 2020. Microsoft maintains it. It receives regular updates and improvements. If we look at the number of downloads for similar frameworks which have been on the market for a while, you can see Playwright has burst onto the scene.

Key Features:

Support for cross-browser platforms built on Chromium, WebKit, and Firefox – which includes Chrome, Edge, Firefox, Opera, and Safari.
Support cross-platform execution on Windows, Linux, and macOS.
Languages supported are JavaScript & TypeScript, Python, .NET, C#, and Java.
Auto-wait built-in, smart assertions that retry until the element is found, and test data tracing – keep track of logs, videos and screenshots easily.
Built with modern architecture and no restrictions – interact with multi-page, multi-tab websites like a real user, and tackle frames and browser events with ease.
Run test parallelly and it runs faster than another automation tool.
Integrate POMs as extensible fixtures.

Disadvantages:

It is new so APIs are evolving.
It has no support for IE browser and Native Mobile Apps.
Community support is not so good, but yes they are improving.

Overview

Setup

It is simple to set up Playwright in your project or create a new test project.

Ensure you have NodeJS installed.
Install a code editor. We will be using Visual Studio Code.

Installing

You can scaffold your project using the init command.

# Run from your project's root directory
npm init playwright@latest

# Or create a new project
npm init playwright@latest new-project

To add dependency and install browsers.

# install supported browsers
npx playwright install

The above command installs the Playwright version of Chromium, Firefox, and Webkit browser.

Writing Tests

Automatic test generation

Playwright also ships with a code generator that can generate the tests for you. Run codegen and interact with the browser. The playwright will generate the code for the user interactions. codegen will attempt to build text-based selections that are durable.

You can copy generated code by clicking the button near the record button.

Write Code

Create for TypeScript to define your test.

import { test, expect } from '@playwright/test';
test('basic test', async ({ page }) => {
await page.goto('https://playwright.dev/');
const title = page.locator('.navbar__inner .navbar__title');
await expect(title).toHaveText('Playwright');
});

Executing Tests

Execute the Playwright test using the below command. Assuming that test files are in the directory.

Run all the tests:

npx playwright test

Run a single test file:

npx playwright test tests/example.spec.ts

Playwright test just runs a test using Chromium browser, in a headless manner. Let's tell it to use the headed browser:

npx playwright test --headed

Run all the tests against a specific project:

npx playwright test --project=chromium

There are more commands to run the test as per requirement, you can see them - Playwright command line.

Create maintainable automated tests

Page Object Model

Page Object Model is a common pattern that introduces abstractions over web app pages to simplify interactions with them in multiple tests.

We will create a helper class to encapsulate common operations of the page. Internally, it will use the object.

// playwright-dev-page.ts
import { expect, Locator, Page } from '@playwright/test';

export class PlaywrightDevPage {
  read-only page: Page;
  readonly getStartedLink: Locator;
  readonly gettingStartedHeader: Locator;
  readonly pomLink: Locator;
  readonly tocList: Locator;

  constructor(page: Page) {
    this.page = page;
    this.getStartedLink = page.locator('a', { hasText: 'Get started' });
    this.gettingStartedHeader = page.locator('h1', { hasText: 'Getting started' });
    this.pomLink = page.locator('li', { hasText: 'Playwright Test' }).locator('a', { hasText: 'Page Object Model' });
    this.tocList = page.locator('article div.markdown ul > li > a');
  }

  async goto() {
    await this.page.goto('https://playwright.dev');
  }

  async getStarted() {
    await this.getStartedLink.first().click();
    await expect(this.gettingStartedHeader).toBeVisible();
  }

  async pageObjectModel() {
    await this.getStarted();
    await this.pomLink.click();
  }
}

Now we can use the class in our tests.

// example.spec.ts
import { test, expect } from '@playwright/test';
import { PlaywrightDevPage } from './playwright-dev-page';

test('getting started should contain table of contents, async ({ page }) => {
  const playwrightDev = new PlaywrightDevPage(page);
  await playwrightDev.goto();
  await playwrightDev.getStarted();
  await expect(playwrightDev.tocList).toHaveText([
    'Installation',
    'First test',
    'Configuration file',
    'Writing assertions',
    'Using test fixtures',
    'Using test hooks',
    'VS Code extension',
    'Command line',
    'Configure NPM scripts',
    'Release notes'
  ]);
});

test('should show Page Object Model article', async ({ page }) => {
  const playwrightDev = new PlaywrightDevPage(page);
  await playwrightDev.goto();
  await playwrightDev.pageObjectModel();
  await expect(page.locator('article')).toContainText('Page Object Model is a common pattern');
});

Run Tests against different environment

In many projects, we maintain different environments like dev, QA and prod, etc. To make environment variables easier to manage, consider something like .env files. Here is an example that uses the dotenv package to read environment variables directly in the configuration file.

Global Setting for pick env file to run a test against:
In the playwright.config.ts file add the below code.

// playwright.config.ts
import type {
    PlaywrightTestConfig
} from '@playwright/test';
import {
    devices
} from '@playwright/test';

//ADD THIS CODE
import dotenv from 'dotenv';

//ADD THIS CODE
if (process.env.test_env) {
    // Alternatively, read from env file.
    dotenv.config({
        path: `.env.${process.env.test_env}`,
        override: true
    })
} else {
    dotenv.config({
        path: `.env.development`,
        override: true
    })
}

/**
 * See https://playwright.dev/docs/test-configuration.
 */
const config: PlaywrightTestConfig = {
    testDir: './tests',
    timeout: 30 * 1000,
    globalSetup: 'tests/global-setup.ts',
    use: {

        // READ REDIRECT_URI FROM THE ENVIRONMENT FILE
        baseURL: process.env.REDIRECT_URI,
        storageState: 'storageState.json'
        actionTimeout: 0,
        trace: 'on-first-retry',
    },
    expect: {
        timeout: 5000
    },
    fullyParallel: true,
    forbidOnly: !!process.env.CI,
    retries: process.env.CI ? 2 : 0,
    workers: process.env.CI ? 2 : undefined,
    reporter: 'html',
    /* Configure projects for major browsers */
    projects: [{
        name: 'chromium',
        use: {
            ...devices['Desktop Chrome'],
            baseURL: process.env.REDIRECT_URI,
            // Tell all tests to load the signed-in state from 'storageState.json'.
            storageState: 'storageState.json'
        },
    }, ],

};

export default config;

Pass environment in test command:

‍In package.json add the below commands to the script section. Install the cross-env package which Runs scripts that set and use environment variables across platforms.

// package.json 
    "test-dev": "cross-env test_env=development npx playwright test",
    "test-qa": "cross-env test_env=qa npx playwright test",
    "test-prod": "cross-env test_env=prod npx playwright test"

Environment Files:

‍We have environment files like the below .env.prod file. Same REDIRECT_URI key added in .env.development and .env.qafiles but with different values.

//.env.prod
REDIRECT_URI=https://app.redpen.ai/

Run test against the environment :

‍Now run a test against the different environments by running commands as per env.

// FOR DEV
    npm run test-dev

    // FOR QA
    npm run test-qa

    // FOR PROD
    npm run test-prod

    // YOU CAN PASS OTHER OPTIONS BY ADDING --
    npm run test-prod -- --headed

Parameterization of tests

Like Selenium Playwright also offers parametrized tests. You can either parametrize tests on a test level or on a project level.

Test Level:
You can also do it with the test.describe() or with multiple tests as long the test name is unique.

// example.spec.ts
    const people = ['Alpha', 'Beta'];
    for (const name of people) {
      test(`testing with ${name}`, async () => {
        // ...
      });
    }

Project Level:

Playwright Test supports running multiple test projects at the same time. In the following example, we'll run two projects with different options.

We declare the option person and set the value in the config. The first project runs with the value Alpha and the second with the value Beta.


  // my-test.ts
    import { test as base } from '@playwright/test';

    export type TestOptions = {
      person: string;
    };
    export const test = base.extend >TestOptions >({
      // Define an option and provide a default value.
      // We can later override it in the config.
      person: ['John', { option: true }],
    });

Now in the test file, we can use it.

// example.spec.ts
    import { test } from './my-test';

    test('test 1', async ({ page, person }) => {
      await page.goto(`/index.html`);
      await expect(page.locator('#node')).toContainText(person);
      // ...
    });

Now, we can run tests in multiple configurations by using projects. We can also use the option in a fixture.

// playwright.config.ts
    import type { PlaywrightTestConfig } from '@playwright/test';
    import { TestOptions } from './my-test';

    const config: PlaywrightTestConfig <TestOptions > = {
      projects: [
        {
          name: 'alpha',
          use: { person: 'Alpha' },
        },
        {
          name: 'beta',
          use: { person: 'Beta' },
        },
      ]
    };
    export default config;

Writing Modular Tests

When testing complex, integrated applications, test teams frequently turn to "modular testing" as a way to break down application functionality into small pieces. The modular pattern provides an easier-to-follow road map and then rearranges the chunks of functionality into software testing scenarios that represent different customer workflows. Creating a modular regression suite takes time, but in the end, the business has a complete list of functional areas, along with integration points.

A detailed Blog on the Modular test is here Modular Testing.

Running automation tests from CI/CD

GitHub Actions

It is very easy to run the Playwright automation tests using the GitHub action. Like all the other GitHub actions, you need to create a YAML file for the execution.

An Example (Worth a thousand words)

Here is a sample YAML file that can help you to get started. Please go through each step carefully.

# Provide a name of the action that will appear in the actions
  name: Playwright Tests
  on:
    # The pipeline will be executed when a changeset is pushed to one of the branches defined here
    push:
      branches: [ 'dev' ]
    # The pipeline will be executed when a changeset is pushed to a pull request created against one of the branches defined here
    pull_request:
      branches: [ 'dev' ]
  jobs:
    test:
      timeout-minutes: 60
      runs-on: ubuntu-latest
      steps:
      - uses: actions/checkout@v2

      # Install the required node version by the Playwright and your project
      - uses: actions/setup-node@v2
        with:
          node-version: '14.x'

      # Install the node dependencies for your project
      - name: Install dependencies
        run: npm ci

      # This will install the browsers on which you want to run the tests
      - name: Install Playwright Browsers
        run: npx playwright install chromium --with-deps

      # This will run the tests along with the configurations defined in this command and your configuration file
      # The result of the tests will be exported according to the configurations.
      # Here, we have set the configuration in a way that the test result will be a JUnit test result.
      # The test result will be exported in a file named results.xml.
      - name: Run Playwright tests
        run: npx playwright test --project=chromium

Azure Pipeline

Create an Azure Pipeline for Playwright Tests

Integration with Test Case Management Tools

The GitHub action or the Azure Pipeline can also publish the test result directly to your test management tool.

Let's see how it can be done for the two most used test management tools in the industry.

Xray Integration

Xray is a test management tool widely used in Jira. It integrates very well in Jira. Test cases, suits, plans, and executions are also created as Jira issues by Xray. Let's see how our CI/CD pipeline can be integrated with the Xray.

Prerequisites

A Jira site having Xray installed.
Xray client id and client secret.
Xray test plan.

YAML file changes

# Here, we are going to use the APIs of Xray to report the test result.
    # The first API call is made to get an Xray token.
    # You will require an Xray client id and client secret to make this API call. Your Jira site admin can generate them.
    # Once you have them, store them in the GitHub secrets.
     You will require a client id and client secret for the same.
    - name: Get XRay Token
      id: tokenRequest
      uses: fjogeleit/http-request-action@v1
      with:
        url: 'https://xray.cloud.getxray.app/api/v1/authenticate'
        method: 'POST'
        customHeaders: '{"Content-Type": "application/json"}'
        data: '{"client_id":"${{ secrets.XRAY_CLIENT_ID }}","client_secret":"${{ secrets.XRAY_CLIENT_SECRET }}"}'
        timeout: 30000

    # This API call is made to publish a test execution to the Xray test plan.
    # Please replace your_project_key and your_test_plan_id with your actual values in the URL below
    # If the API call is successful, you will be able to see a new test execution in the test plan mentioned in the URL
    - name: Post the test results to XRay
      uses: fjogeleit/http-request-action@v1
      with:
        url: 'https://xray.cloud.getxray.app/api/v1/import/execution/junit?projectKey=your_project_key&testPlanKey=your_test_plan_id'
        method: 'POST'
        customHeaders: '{"Content-Type": "application/xml"}'
        bearerToken: ${{ fromJSON(steps.tokenRequest.outputs.response) }}
        timeout: 30000
        file: 'results.xml'