The Testing Academy · Class Notes Monday, 7 September (IST)
Live class · study guide

Three layers, and a spec that is only assertions

The CRUD flow worked, but it was not scalable: every spec rebuilt the base URL, the headers and the payload by hand. Today that gets pulled apart into three layers, an API helper for the request, a service object plus a fixture for the payload and the token, and a spec that holds nothing but assertions. Then JSONPath for reading responses, and one honest warning about letting AI optimise your framework.

By Pramod Dutta, The Testing Academy. Study notes from the live Playwright 2x class, rebuilt from the session recording, the batch repository and the Eraser deck. The code here was read from the commits pushed during the class rather than transcribed from the screen, and every status code in the table was measured against the live RESTful Booker API before publishing, not quoted from memory. Two things the repository does that the session did not show are flagged in place.

01

Where this sits

The previous class got a full create, update and delete flow working against RESTful Booker with typed interfaces. It worked, and it was not scalable.

The problem is visible the moment you write a second spec. Every one of them was rebuilding the same things by hand:

  • the base URL
  • the headers
  • the payload
  • the request.post call itself

All of that is common. None of it is the test.

02

Three parts, three homes

Every API spec has three parts: arrange, act, assert. Today each one gets its own file, and once they do, the spec shrinks to the only part that is actually about testing.

ASSERT the spec filecreate-booking.spec.ts, booking-crud.e2e.spec.ts ARRANGE, the payload and the token BookingApi.tspayloads, one method per call booker.fixture.tshands over the token ACT, the request itself ApiHelper.tsbuildUrl, callApi, get, post, put, patch, delete What moved where base URLheadersmethod dispatchTO THE HELPER payload buildingtokenserialise, deserialiseTO THE SERVICE OBJECT expect(...)STAYS IN THE SPEC
The spec calls the fixture and the service object; the service object calls the helper. Nothing calls upward.

This is the same shape as the web side of the framework. There, page objects hold the interactions and the spec holds the assertions. BookingApi is a page object for an API, and it earns its place for the same reason: the thing that changes when the API changes is in one file.

03

Layer one: the API helper

src/utils/ApiHelper.ts is a plain class. It takes a context in its constructor, which can be a Page or an APIRequestContext, and everything else hangs off that.

The two methods that matter:

TypeScript
private buildUrl(url: string, params?: Record<string, string>): string {
    if (!params) return url;
    const searchParams = new URLSearchParams(params);
    return `${url}?${searchParams.toString()}`;
}

async callApi(options: ApiRequestOptions): Promise<APIResponse> {
    const { url, method, headers, data, params, timeout } = options;
    const request = this.getRequest();
    const fullUrl = this.buildUrl(url, params);

    switch (method) {
        case 'GET':    return await request.get(fullUrl, { headers, timeout });
        case 'POST':   return await request.post(fullUrl, { headers, data, timeout });
        case 'PUT':    return await request.put(fullUrl, { headers, data, timeout });
        case 'DELETE': return await request.delete(fullUrl, { headers, timeout });
        case 'PATCH':  return await request.patch(fullUrl, { headers, data, timeout });
        default:
            throw new Error(`Unsupported HTTP method: ${method}`);
    }
}

callApi is the only method that actually issues a request. Everything else is a thin wrapper over it:

TypeScript
async post(url: string, data?: unknown, options?: Omit<ApiRequestOptions, 'url' | 'method' | 'data'>) {
    return this.callApi({ url, method: 'POST', data, ...options });
}

Read that Omit carefully, because it is doing real work. The caller cannot pass url, method or data a second time through options, since those are already positional. What they can still pass is extra headers, params or a timeout, which is how a per-call authorization header gets in later.

The class also carries:

Member What it does
callApiWithRetry Repeats the call until a condition passes. Defaults: 3 attempts, 5000 ms apart
parseJsonResponse response.json() with a type parameter, so the body is typed at the call site
isSuccess status in the 200 range
isFailureClient status in the 400 range

Asked in class: can we use Playwright's own retries option instead? Yes, and they solve different problems. Playwright's retry re-runs the whole test. callApiWithRetry re-issues one request while the test keeps running, which is what you want for an endpoint that is eventually consistent.

04

What a spec looks like once arrange is gone

Before, a create-booking test set the base URL, built headers, assembled a payload and called request.post. After, the arrange half is one line and the act half is one line:

TypeScript
const { bookingid, booking } = await bookingApi.createBooking(payload);

expect(bookingid).toBeGreaterThan(0);
expect(booking.firstname).toBe('E2E');

The response body is also attached to the HTML report, which costs one call and pays for itself the first time a test fails in CI:

TypeScript
await testInfo.attach('created-booking', {
    body: JSON.stringify({ bookingid, booking }, null, 2),
    contentType: 'application/json',
});

Run it with the dedicated API project, which launches no browser at all:

Terminal
npx playwright test src/tests/apisTests --project=api
05

Layer two: the service object and the fixture

src/api/BookingApi.ts holds every payload and every endpoint. Each operation becomes a method, and each method has two forms:

  • createBookingResponse(payload) returns the raw response, so a test can assert on a status code.
  • createBooking(payload) returns the typed body, and throws if the call failed.

That pairing is deliberate. A typed method that returned a Booking-shaped object after a 404 would be lying to you, so it throws instead. The negative spec asserts exactly that:

TypeScript
await expect(bookingApi.getBooking(99_999_999)).rejects.toThrow(/failed: 404/);

The fixture is short, and it is the piece that makes the specs readable:

TypeScript
export const test = base.extend<BookerFixtures>({
    bookingApi: async ({ request }, use) => {
        await use(new BookingApi(request));
    },
    bookerToken: async ({ bookingApi }, use) => {
        await use(await bookingApi.getToken());
    },
});

Now any test that writes async ({ bookingApi, bookerToken }) in its signature is handed a ready service object and a live token. No setup, no beforeAll.

06

The token, and the status code that surprises everyone

Someone asked what happens when the token expires. The answer in the repository is more careful than "make a new one".

Cached tokenminted on first use Send the requestCookie: token=... 403?not 401 Donereturn the response Re-authenticate once, then retrygetToken(true) drops the cache and mints a new token no yes
The retry happens once. A second 403 is a real failure and is allowed to surface.

Two lines carry the whole design:

TypeScript
async getToken(forceRefresh = false): Promise<string> {
    if (forceRefresh) this.invalidateToken();
    return (this.cachedToken ??= await this.auth());
}

??= is logical nullish assignment: assign only if the left side is null or undefined. So the first call authenticates and caches, and every later call returns the cache. That single expression replaced a longer block during the class, which is covered in the Ponytail section below.

The retry itself:

TypeScript
private async sendAuthed(
    send: (token: string) => Promise<APIResponse>,
    explicitToken?: string,
): Promise<APIResponse> {
    if (explicitToken !== undefined) return send(explicitToken);
    const res = await send(await this.getToken());
    return res.status() === TOKEN_REJECTED ? send(await this.getToken(true)) : res;
}

The explicitToken branch matters more than it looks. If a test passes a token deliberately, it goes out untouched, including a deliberately bad one, so a negative test can still assert a 403. Only the managed token renews itself. A helper that silently fixed every bad token would make that test unwritable.

Every status code below was measured against the live API before this page was published, not recalled:

Call Status The part that catches people
POST /auth with correct credentials 200 Token is 15 characters
POST /auth with wrong credentials 200 Body is {"reason":"Bad credentials"}. The status is a success
POST /booking 200 Not 201, despite creating something
PUT /booking/{id} with a bad token 403 Not 401
PUT /booking/{id} with no token at all 403 Same code as a bad token
DELETE /booking/{id} with a valid token 201 Created, on a delete
GET /booking/{id} after deleting 404

The second row is the one that will bite you. A test that checks only the status code will pass while authentication is completely broken, which is why the repository checks the body:

TypeScript
const body = await this.apiHelper.parseJsonResponse<{ token?: string; reason?: string }>(response);
if (!body.token) {
    throw new Error(`[BookingApi] /auth returned no token. Reason: ${body.reason ?? 'unknown'}`);
}

Authorization on this API travels in a Cookie header, Cookie: token=..., not in Authorization: Bearer. That is a quirk of RESTful Booker, not a general rule, and it is the reason authHeaders exists as its own small method.

07

Sharing a booking id across tests

The lifecycle spec creates in one test, updates in the next and deletes in the third. The id is shared through a variable in the describe scope:

TypeScript
test.describe.serial('@e2e @P0 Level 3 - Booking lifecycle (token from fixture)', () => {
    let bookingId: number;
    ...
});

describe.serial is what makes that safe. It forces the tests to run in order, and if one fails the rest are skipped rather than running against an id that was never created.

Asked twice in class, in different words: how do I pass data from one test to another? This is the answer. Note the trade-off, though: serial tests cannot run in parallel with each other, so keep serial blocks small and put genuinely independent checks in their own tests.

08

Test data, two ways

The payload builder lives in src/testdata/booking.data.ts and draws every value from one utility:

TypeScript
export function buildBookingFromGenerator(
    overrides: Partial<Booking> = {},
    stayNights = 3,
): Booking {
    const checkin = DataGenerator.dateOffset(1);
    return {
        firstname: DataGenerator.firstName(),
        lastname: DataGenerator.lastName(),
        totalprice: DataGenerator.number(100, 1000),
        depositpaid: DataGenerator.bool(),
        bookingdates: {
            checkin,
            checkout: DataGenerator.dateOffset(stayNights, new Date(checkin)),
        },
        additionalneeds: DataGenerator.oneOf(ADDITIONAL_NEEDS),
        ...overrides,
    };
}

Faker and the in-house DataGenerator both work, and the class showed the swap live. The repository standardised on DataGenerator so that all randomness comes from one place, with @faker-js/faker still installed behind it.

Something the repository fixed that the class did not see. The commit fix: stop buildBooking inverting the stay dates landed after the walkthrough. Check-out is now derived from check-in, so the stay is always the right way round, and check-in is relative to today so a booking never ages into the past. If you are copying the version from the recording, take this one from the repository instead.

09

Negative tests

The AI was asked to add negative coverage, and the useful shape it produced is a table-driven block: a list of [name, call, expectedStatus] rows, each turned into a test.

What is covered:

  • GET a booking id that does not exist, expect 404
  • POST /booking with an empty payload
  • POST /booking with a malformed payload
  • POST /auth with the wrong username and password
  • PUT with an invalid token, expect 403

The last one is the best-written test in the file, because it asserts on more than the status:

TypeScript
expect(response.status()).toBe(403);
...
expect((await bookingApi.getBooking(bookingid)).firstname).not.toBe('ShouldNotStick');

A 403 tells you the request was rejected. Reading the booking back tells you nothing changed. Those are different claims, and only the second one proves the authorization actually held.

10

JSONPath

Assertions so far have been direct property reads. That is fine when you know the shape and it is shallow. JSONPath earns its place when the shape is deep, repeated, or unknown.

The package is jsonpath-plus:

Terminal
npm install -D jsonpath-plus
TypeScript
import { JSONPath } from 'jsonpath-plus';

const result = JSONPath({ path: '$.booking.bookingdates.checkin', json: body });

The one rule that trips everybody up: JSONPath() always returns an array of matches, even when there is exactly one. A single-value read ends in [0], or you pass wrap: false.

THE DOCUMENT store book [ 0, 1, 2, 3 ] author title price category bicycle color price $the root, every path starts here .keya direct child$.store.bicycle.color ..keyat any depth$..price [n] [a:b]index, and slice$.store.book[0:2] *every child$.store.book[*].author
The operators, and the shape they walk. A filter narrows a set: it does not reach deeper.

The filter is where it stops being a convenience and starts being useful. @ is the current item:

Path Returns
$.store.book[?(@.price<10)] every book under 10
$.store.book[?(@.category=='fiction')] every fiction book
$.store.book[?([email protected])] every book with no ISBN
$..author every author anywhere in the document
$.store.book[(@.length-1)] the last book

A full cheat sheet was written during the class as jsonpath-cheatsheet.md, sitting next to a store.json fixture, and the queries in it are exercised by a live spec so the documentation cannot drift from what actually runs.

Asked in class: does this work in REST Assured too? Yes. The concept is the same and Java has its own JSONPath library, though the syntax is not identical: REST Assured uses a Groovy-style path such as store.book.findAll { it.category == 'fiction' }.price rather than a bracketed filter.

11

Ponytail, and when to let AI optimise

The observation behind this: ask an AI for a function and you often get a class, a constructor, some initialisation, and then the function. Fifty lines where one would do.

Ponytail (ponytail.dev) is an open source tool that pushes the agent the other way. It works with Claude Code, GitHub Copilot, Codex and others, and it needs a session restart after installing, which is exactly what went wrong the first time it was tried live.

Applied to the token code, it collapsed the cache handling into the ??= expression shown earlier. Same behaviour, far fewer lines.

The advice attached to it was firm, and it is the most important thing in this section:

Use Ponytail only when your framework is already 100% done. Optimised code is harder to read, and if you optimise while you are still learning the structure you will lose track of where things happen. The related rule, repeated as a request rather than an instruction: build the base framework by hand at least three times before you let AI generate your utilities. After that, use AI freely.

And one caveat from the room, accepted rather than argued with: optimisation tools can also make mistakes, so know what you are doing when you accept the change.

A second thing the repository hit that the session did not show. A spec failed with Cannot find module '@/utils/ApiHelper'. Two separate bugs behind one message: the alias @/* does not exist in this project (it defines @api/*, @config/*, @fixtures/*, @pages/*, @testdata/*, @utils/*), and two files were on disk with different capitalisation than their imports. The second bug is the dangerous one. macOS is case-insensitive, so it passes locally and fails only on a Linux CI runner. A case-only rename also needs two steps through a temporary name, or Git records nothing.

12

Tasks and announcements

This week's task:

  • Add the API helper and the fixtures to your own framework.
  • Write a complete end-to-end RESTful Booker API test using both, covering create, update and delete.
  • Add three or four negative scenarios alongside it.

Next class

  • JSON schema validation, which was started and then deliberately deferred so it gets proper time.
  • An AI data generator wired into the framework.
  • After that the framework is complete, and it goes to Jenkins, Docker and GitHub Actions.

Announcements

  • Sharding and parallel execution come after the Jenkins sessions. Parts 3 and 4 of Jenkins are still outstanding, then GitHub Actions, then sharding.
  • Two self-paced Cucumber videos are being shared for anyone who wants to add Cucumber to this framework. It is optional.
  • Extra certification sessions on Tuesday and Thursday evenings. Invitations to follow.
  • The Cucumber walkthrough is planned for Wednesday evening.