sebee.website🌱
  • Articles
  • CV
  • About
  • Contact

Sebastian Pieczynski's website and blog

Published on: yyyy.mm.dd

Back to Articles
sebee.website
  • Created with ❤️ by Sebastian Pieczyński © 2023-2025.

Links

  • Terms of Use
  • Privacy Policy
  • Cookie Policy
  • ethernal
  • @spieczynski

Playwright introduction to end to end testing

Published on: 12/7/2023

a dark skin pirate woman, with short dark hair and a wide round hat in a blue low top open back dress with thin vertical blue and teal stripes with a small parrot on her arm sitting behind wooden desk weighting gold coins on an old fashioned scale, camera centered front facing, watercolor

Contents

  1. Introduction
  2. Why test?
  3. Playwright capabilities
  4. Installing Playwright
  5. Playwright introduction
  6. Writing first test
    1. Defining data-testid for elements in the application
    2. Managing test cases
  7. Visual tests
  8. Testing responsive views
  9. Tips
    1. Run tests in UI / "headed" mode
    2. Define viewport for Playwright test generator
    3. Disable reports opening on failed tests
  10. Negating the condition
  11. Gotchas
  12. Conclusion
  13. Reference materials
  14. Thank you!

Introduction

We have finally arrived with our treasures to the hideout. Now's the time to check if what we brought is worth it's weight.

Why test?

Tests are usually the first thing that goes out the window when projects have tight deadlines. When a project is not yet defined and does not have a set structure it may even be understandable but not testing and not automating these tests for engineers at a stage when project is being developed is a waste of everyone's time.

Automated end to end tests allow us to describe what user would do (clicks on buttons, typing, etc) and then check if it works as expected. All this is done by the software that runs the tests. Until recently the leading solution was Cypress but now new kid is on the block and working with it is a joy.

It's called Playwright.

Playwright capabilities

What can Playwright do for us?

  1. Automate manual testing process.
  2. Fill out forms and submit them.
  3. Run tests in parallel.
  4. Run tests in multiple browsers.
  5. Record the way we test the application and create automated test from that interaction.
  6. Take screenshots and compare them for visual regressions.
  7. Run tests with different viewport settings.

That is a lot of great functionality out of the box. Let's see how we can utilize it.

Installing Playwright

To add Playwright to your project invoke:

You should see the prompt in the terminal similar to this one:

We initialize new project in current directory (root of the project), put our tests in tests directory and ignore GitHub actions for now. We also allow Playwright to install browsers so we can test against them.

Playwright introduction

In this part we will reuse the frontend of our scraper that we have built and add end to end tests to it.

First let's take a look at the example Playwright created for us:

There are two tests in this example named:

  • has title - after navigating to https://playwright.dev it checks if page has title Playwright and
  • get started link - checks if the same page has heading with name Installation and a link with name Get started.

We'll use such tests for our website as well.

Writing first test

We'll start by deleting all tests and examples and create our own.

Remove tests-examples folder as well as example.spec.ts file.

In the tests folder create productPage.spec.ts file.

in the package.json add a script to run the tests:

Before running any tests make sure the backend and frontend are running. We can do that by running in terminal(s) the commands:

and

Now let's try to run it:

At this point we could write the rest of the tests manually but let's make our lives a bit easier and see how we can automate even that part.

If you have a VS Code plugin installed use that as it automatically creates files and fills them in IDE. If not you can run the codegen that will bring up the window with the empty browser and Playwright Inspector.

Note that with codegen after recording the test you must copy and paste the code into the test file and/or create the file yourself.

Playwright Codegen Interface

From here you need to input the correct url you want to test:

Clicking the buttons will test for presence, visibility, content and values.

Playwright Codegen Interface

  1. Record (or pause recording) test.
  2. Locator picker.
  3. Visibility picker.
  4. Content picker.
  5. Value picker.

If you are creating the test file yourself name it: productFilter.spec.ts or rename created test-1.spec.ts.

The finished test after cleanup could look something like this:

You will note that dragging the slider does not really work in this case and for fine grained control it needs to be coded manually (with fill) into the test.

Now run the tests:

It should pass with output:

If something is wrong check if the server and frontend are running or contact me.

Defining data-testid for elements in the application

For hard to find elements it is a good practice to add testid to it so it can be looked up easier. Let's do this for our products. Read more in the docs.

In the App.tsx file add the following code just below products map:

The data-testid="product" allows us to later use it as a locator in our tests. Learn more from Playwright documentation.

Managing test cases

Let's separate the created test into more workable chunks and add some more tests.

Here's a new test file. There's an intentional bug here, one of the tests will fail:

Here's a tip: the easies way to debug it is to run the tests in UI / "headed" mode:

You'll also note that for sorting we test the first element displayed before and after the slider change:

Playwright has a lot of functions that we can use to test what happens in our application. Read more in the locator assertion documentation.

More on this in the Tips section below and in the Playwright docs.

Running with the --ui flag will bring up the Playwright Inspector with all the tests that can be run manually. Page is not static, you can use the tools provided in the inspector to see what exactly has happened along with network activity!

Spend some time looking for the bug or scroll below for an answer. Side note: struggling a bit with the code leads to better retention so do spend some time here if you can.

image

If at any point you try to change things and get annoyed by the reports popping up open the playwright.config.ts and change the reporter to:

Setting option open to never will prevent the browser from opening the report and forcing you to kill the process in the terminal.

The bug in this test is the expectation of the Skuntank when it should not be visible at this price range:

Change it to the 'Silcoon' as show below:

Full and corrected test:

Now our tests pass and you learned how to debug them visually.

And talking about visual side. Wouldn't it be nice to check if the website looks the way we expect it to?

Visual tests

Adding visual comparison is a matter of a single assertion. Add this test to the file:

The test will go to the home page with default parameters and verify if the screenshot it produces looks the way we expect it to.

But how do we make a screenshot you might ask.. we don't, Playwright does but it also means it will not find it the first time it runs so there will be errors and these are EXPECTED:

The test will fail but the screenshot is now saved into the test\productFilter.spec.ts-snapshots folder. That folder should be committed to your repository!

Try running the test again:

Now all our tests pass!

You just learned how to do (basic) visual testing with Playwright!

There are more capabilities to this API ex. how different the images can be when compared. Read more here.

A note on screenshots: these will look different than what you can expect in your browser. Open them up and compare. It might be limitation of the way Playwright works with browser engines so it's still valid to test in real browsers.

Testing responsive views

These are all and good, but what about mobile first? Shouldn't that be tested as well?

By all means! The syntax is a bit different to create a test for responsive views as we need to set the viewport size. We'll use the visual testing to make sure that it looks as we expect.

Add the following test to the file:

After the screenshot had been created we can run the test again:

and see that all the tests are passing:

Finally to run the codegen in mobile view you can execute:

It will open the Playwright Inspector with viewport limited to 280px width and 800px height.

Tips

These are the most important bits from this article that you may want to reference later.

Run tests in UI / "headed" mode

When your tests are failing and you have trouble figuring why run them in a ui mode:

The double dash -- will pass arguments to the script invoked that is the --ui part and will run the test in UI mode.

Define viewport for Playwright test generator

Disable reports opening on failed tests

Negating the condition

What if you want to check if something is NOT visible? Playwright does not expose methods such as notVisible. There is a negation operator/property that you can use to look for the opposite of the condition.

See documentation on not property.

Gotchas

You may see an error when writing a test:

TL;DL: Do NOT use more than one goto in a single test invocation with assertions in between.

This will happen always (for me) if there is more than one goto instruction per test. For example if you want to test if all navigation buttons work in one go, but to be certain you also make assertions on the page.

Separate these tests with a test function and you're golden.

Conclusion

Today we have learned:

  1. How to write simple end-to-end tests with Playwright.
  2. How to use testids for tests.
  3. How to test user interactions.
  4. How to test for visual regressions.
  5. How to test responsive views.
  6. How Playwright helps us automate the process of testing and gives us confidence that our product still works (and looks) as expected.

Reference materials

  1. Test assertions
  2. Assertion methods
  3. Screenshot options
  4. Negate assertions with not property
  5. Using testids

Thank you!

This way we have come to the end of our journey. Our coffers secured, tools laying in the shed and treasures in the safe. It was a great journey and I do hope you enjoyed it as much as I have. Thank you for your company!

You can now write basic tests for you application. I do hope to explore this topic in more detail at some point so let me know how you like it!

If no one told you this today: You are a great person and do great work! We all 💖 you! Do not give up!

Back to Articles
1NS_BINDING_ABORTED; maybe frame was detached?
1npm init playwright@latest
1Getting started with writing end-to-end tests with Playwright:
2Initializing project in '.'
3√ Where to put your end-to-end tests? · tests
4√ Add a GitHub Actions workflow? (y/N) · false
5√ Install Playwright browsers (can be done manually via 'npx playwright install')? (Y/n) · true
6Installing Playwright Test (npm install --save-dev @playwright/test)…
example.spec.ts
1import { test, expect } from '@playwright/test';
2
3test('has title', async ({ page }) => {
4 await page.goto('https://playwright.dev/');
5
6 // Expect a title "to contain" a substring.
7 await expect(page).toHaveTitle(/Playwright/);
8});
9
10test('get started link', async ({ page }) => {
11 await page.goto('https://playwright.dev/');
12
13 // Click the get started link.
14 await page.getByRole('link', { name: 'Get started' }).click();
15
16 // Expects page to have a heading with the name of Installation.
17 await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
18});
productPage.spec.ts
1import { test, expect } from '@playwright/test';
2
3test('has title', async ({ page }) => {
4 await page.goto('http://localhost:5173');
5 expect (await page.title()).toBe('Vite + React + TS');
6});
package.json
1 "scripts": {
2 //...
3 "test": "npx playwright test"
4 }
1npm run server
1npm run dev
1npm run test
1npx playwright codegen
1http://localhost:5173
productFilter.spec.ts
1import { expect, test } from '@playwright/test';
2
3test('e2e product page', async ({ page }) => {
4 await page.goto('http://localhost:5173/?price_lte=80');
5 await expect(page.getByText('Max Price:')).toBeVisible();
6 await expect(page.getByText('Sort (by price)')).toBeVisible();
7 await expect(page.getByText('Total Products:')).toBeVisible();
8 await expect(page.getByLabel('Max Price:')).toBeVisible();
9 await page.getByLabel('Max Price:').fill('112');
10 await expect(page.locator('#root')).toContainText('Total Products: 387');
11 await expect(page.getByText('Ivysaur£')).toBeVisible();
12 await expect(page.getByRole('img', { name: 'Ivysaur' })).toBeVisible();
13 await expect(page.locator('div').filter({ hasText: /^Ivysaur£87$/ }).getByRole('paragraph')).toBeVisible();
14 await page.getByLabel('Sort (by price)').check();
15 await expect(page.getByRole('heading', { name: 'Weedle' })).toBeVisible();
16 await expect(page.locator('div').filter({ hasText: /^Weedle£25$/ }).getByRole('paragraph')).toBeVisible();
17 await page.locator('#sortOrder').selectOption('desc');
18 await expect(page.getByRole('heading', { name: 'Skuntank' })).toBeVisible();
19 await expect(page.locator('div').filter({ hasText: /^Skuntank£112$/ }).getByRole('paragraph')).toBeVisible();
20});
1npm run test
1Running 6 tests using 2 workers
2 6 passed (25.2s)
App.tsx
1{products?.map((data) => {
2 const metadata = JSON.parse(data.data);
3 return (
4 <div key={data.url} data-testid="product" className='border-slate-300 bg-slate-600 border-2 p-4 flex flex-col gap-2'>
5 {/* .. */}
6 </div>
7 )}
8 )
9{/* .. */}
10}
productFilter.spec.ts
1import { expect, test } from '@playwright/test';
2
3test('interface is present and working', async ({ page }) => {
4 await page.goto('http://localhost:5173/?price_lte=80');
5 await expect(page.getByText('Max Price:')).toBeVisible();
6 await expect(page.getByText('Sort (by price)')).toBeVisible();
7 await expect(page.getByText('Total Products:')).toBeVisible();
8 await expect(page.getByLabel('Max Price:')).toBeVisible();
9 await expect(page.locator('#root')).toContainText('Total Products: 258');
10});
11
12test('filtering limits or adds products', async ({ page }) => {
13 await page.goto('http://localhost:5173/?price_lte=80');
14 await expect(page.locator('#root')).toContainText('Total Products: 258');
15 await page.getByLabel('Max Price:').fill('120');
16 await expect(page.locator('#root')).toContainText('Total Products: 405');
17});
18
19test('sorting changes the order of products', async ({ page }) => {
20 await page.goto('http://localhost:5173/?price_lte=80');
21 await expect(page.getByTestId('product').first()).toContainText('Blastoise');
22 await page.getByLabel('Sort (by price)').check();
23 console.log(await page.getByTestId('product').first());
24 await expect(page.getByTestId('product').first()).toContainText('Weedle');
25 await expect(page.locator('div').filter({ hasText: /^Weedle£25$/ }).getByRole('paragraph')).toBeVisible();
26 await page.locator('#sortOrder').selectOption('desc');
27 await expect(page.getByRole('heading', { name: 'Skuntank' })).toBeVisible();
28 await expect(page.locator('div').filter({ hasText: /^Skuntank£112$/ }).getByRole('paragraph')).toBeVisible();
29});
1npm run test -- --ui
productFilter.spec.ts
1await expect(page.getByTestId('product').first()).toContainText('Blastoise');
productFilter.spec.ts
1test('sorting changes the order of products', async ({ page }) => {
2 //...
3 await expect(page.getByRole('heading', { name: 'Skuntank' })).toBeVisible();
4 await expect(page.locator('div').filter({ hasText: /^Skuntank£112$/ }).getByRole('paragraph')).toBeVisible();
5});
productFilter.spec.ts
1test('sorting changes the order of products', async ({ page }) => {
2 //...
3 await expect(page.getByRole('heading', { name: 'Silcoon' })).toBeVisible();
4 await expect(page.locator('div').filter({ hasText: /^Silcoon£80$/ }).getByRole('paragraph')).toBeVisible();
5});
productFilter.spec.ts
1import { expect, test } from '@playwright/test';
2
3test('interface is present and working', async ({ page }) => {
4 await page.goto('http://localhost:5173/?price_lte=80');
5 await expect(page.getByText('Max Price:')).toBeVisible();
6 await expect(page.getByText('Sort (by price)')).toBeVisible();
7 await expect(page.getByText('Total Products:')).toBeVisible();
8 await expect(page.getByLabel('Max Price:')).toBeVisible();
9 await expect(page.locator('#root')).toContainText('Total Products: 258');
10});
11
12test('filtering limits or adds products', async ({ page }) => {
13 await page.goto('http://localhost:5173/?price_lte=80');
14 await expect(page.locator('#root')).toContainText('Total Products: 258');
15 await page.getByLabel('Max Price:').fill('120');
16 await expect(page.locator('#root')).toContainText('Total Products: 405');
17});
18
19test('sorting changes the order of products', async ({ page }) => {
20 await page.goto('http://localhost:5173/?price_lte=80');
21 await expect(page.getByTestId('product').first()).toContainText('Blastoise');
22 await page.getByLabel('Sort (by price)').check();
23 console.log(await page.getByTestId('product').first());
24 await expect(page.getByTestId('product').first()).toContainText('Weedle');
25 await expect(page.locator('div').filter({ hasText: /^Weedle£25$/ }).getByRole('paragraph')).toBeVisible();
26 await page.locator('#sortOrder').selectOption('desc');
27 await expect(page.getByRole('heading', { name: 'Silcoon' })).toBeVisible();
28 await expect(page.locator('div').filter({ hasText: /^Silcoon£80$/ }).getByRole('paragraph')).toBeVisible();
29});
productFilter.spec.ts
1test('visual regressions', async ({ page }) => {
2 await page.goto('http://localhost:5173/?price_lte=80');
3 await expect(page).toHaveScreenshot('productFilter-with-price-80.png');
4});
1[webkit] › productFilter.spec.ts:31:1
2› visual regressions
3
4Error: A snapshot does not exist at
1npm run test
115 passed (40.7s)
productFilter.spec.ts
1 test.describe('page mobile (w-280) screenshot', () => {
2 test.use({ viewport: { width: 280, height: 1200 } });
3
4 test('mobile view about page screenshot', async ({ page }) => {
5 await page.goto('http://localhost:5173/?price_lte=80');
6 await expect(page).toHaveScreenshot('mobile-280productFilter-with-price-80.png');
7 });
8});
1npm run test
118 passed (39.2s)
1npx playwright codegen --viewport-size=280,800 http://localhost:5173/?price_lte=80
1npm run test -- --ui
1npx playwright codegen --viewport-size=280,800 http://localhost:5173/?price_lte=80
1await expect(locator).not.toContainText('error');
playwright.config.js
1export default defineConfig({
2 testDir: './tests',
3 //...
4 /* Reporter to use. See https://playwright.dev/docs/test-reporters */
5 reporter: [['html', { open: 'never' }]],
6 /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
7 use: {
8 //...
9 }
10});
playwright.config.js
1export default defineConfig({
2 testDir: './tests',
3 //...
4 /* Reporter to use. See https://playwright.dev/docs/test-reporters */
5 reporter: [['html', { open: 'never' }]],
6 /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
7 use: {
8 //...
9 }
10});