Playwright MCP + Advance Playwright Framework Creation with Playwright AI Agents & CLI

Table of Contents

• 01 · What is an LLM, really? | The mental model. Next-token weights. Vocabulary you actually need.
• 02 · Where LLMs hit a wall | Six honest limitations that make raw prompting insufficient.
• ★ Bridge · From LLM to AI Agent (and why MCP matters) | Brain vs brain+memory+tools, with the formula on the whiteboard.
• 03 · The Model Context Protocol | MCP as a USB hub for AI. Three roles, the handshake, the tools.
• 04 · Hands-on: Playwright MCP × VWO | Invalid-login flow on app.vwo.com. Zero typed code.
• 05 · The architecture.md blueprint | The 8-layer Pages + Modules contract that turns a prompt into a framework.
• 06 · Custom Reporter (CustomTTAReporter) | Capture every event. Emit your own HTML, JSON, Slack ping.
• 07 · Playwright Test Agents — Planner, Generator, Healer | The official agents. One init-agents command. Three roles.
• 08 · Playwright CLI + CodeGen | The 43 commands · VS Code extension · ~4× cheaper than MCP.

01 · What is an LLM, really?

A Large Language Model is a statistical reader-writer. It has read a lot of the internet, compressed that reading into hundreds of billions of weights, and given a prompt it predicts the next most-likely token, again and again, until a stop signal arrives.

That is the entire trick. There is no database lookup, no live browser, no knowledge base. Everything else — chat memory, tools, web access, “agents” — is scaffolding we build around that core loop.

For a tester, the most useful framing is this: an LLM is a brilliant intern. Confident, articulate, well-read, and unable to verify anything it claims unless we give it a way to check.

How “next token” actually works

The headline claim — “predicts the next most-likely token” — is easier to feel with one concrete example. Take the phrase below. The LLM looks at every word in its vocabulary (~50,000 of them) and assigns each a probability. The token closest to 1.0 wins:

Lower the temperature and the model picks mat almost every time. Raise it, and it starts choosing floor or table sometimes — the same weights, just read with more randomness. Either way, there is no understanding here. There are weights.

The vocabulary you actually need

One round-trip, visualised

02 · Where LLMs hit a wall

A model that has read the internet is still a model that cannot do anything in the world. It cannot click a button, read a database row, or know what time it is. Pretending otherwise is the source of most bad demos.

Six honest limitations

1. No live world. Without a tool, the model can’t browse, query or execute.
2. Hallucinations. It will confidently invent selectors, library APIs and URLs. Test code makes hallucinations easy to catch — they fail fast.
3. Knowledge cutoff. Anything released after training is invisible. New Playwright APIs, your last sprint’s DOM — none of it is in the weights.
4. No memory. Each request starts blank. State must be carried in the prompt or persisted by the surrounding runtime.
5. Limited context. A whole repo will not fit. You must choose what to show — itself a skill.
6. Cannot verify itself. It cannot run the test it just wrote. It needs an executor to close the loop. That executor is the entire point of MCP.

Every limitation in that list is a gap you can fill with a tool. — The reason this whole document exists.

What this means for testers

Testing is a closed-loop discipline. You write a check, run it, observe the result, and decide. An LLM alone can only do the first step. The remaining three — run, observe, decide — require something on the other end of a tool call. For us that thing is a browser, driven by Playwright, exposed through MCP.

Bridge — From LLM to AI Agent (and why MCP matters)

An LLM by itself is just a brain in a jar — brilliant, but trapped. Add memory and tools, and the brain becomes an agent: something that can act in the world, remember what it did, and decide what to do next.

The formula on the whiteboard

The brain-and-body analogy

Picture two robots. One has a head — and nothing else. It can think, but it can’t do. The other has a head, a body, hands holding tools, and legs to walk. Same brain in both. The difference is what surrounds it.

So… why do MCPs matter?

Once you accept that an agent needs tools, the next question is: how do you wire those tools up? The hard way is to write custom integration code for every tool the agent might need — a bespoke browser library, a bespoke file API, a bespoke database client. Every host (Claude Code, Cursor, Claude Desktop) has to redo the work for every tool. It does not scale.

MCP solves this with one standard protocol. Write a tool server once; every MCP-aware agent can use it. Ship a Playwright MCP server, and Claude Code, Cursor and Claude Desktop all get browser automation for free. Switch from Claude to GPT — your servers keep working.

03 · The Model Context Protocol (MCP)

MCP is an open protocol — JSON-RPC over stdio or HTTP — that lets any LLM client talk to any tool server in a uniform way. A Playwright MCP server exposes the browser; a database MCP server exposes SQL; a filesystem MCP server exposes your repo. The model picks tools by name, the runtime executes them.

The one-line analogy: MCP is a USB hub for AI

Before USB, every peripheral had its own cable. Printer port. Mouse port. Modem port. Each cable, each driver, each shape — bespoke. USB collapsed that mess into one shared socket: any device, one cable, plug-and-play.

MCP is the USB hub for AI tools. Slack, Jira, your database, Playwright — each is a different “peripheral.” MCP is the one socket the LLM speaks. Plug a server in, and the model can use it without anyone writing custom glue.

Three roles, one handshake

• Host — the application the user sits in front of (Claude Desktop, Cursor, Claude Code, your IDE).
• Client — a connector inside the host that speaks MCP to one server.
• Server — a process that exposes tools, resources and prompts. Playwright ships one of these.

What a server can offer

Sequence of one tool call

The only config you really need

The host holds a small JSON file that lists the servers it should launch. For Claude Code, this lives in .mcp.json at your project root.

.mcp.json// Tells the host which MCP servers to start for this project.
{
  "mcpServers": {
    "playwright": {
      "command": "npx",
      "args": ["-y", "@playwright/mcp@latest"]
    }
  }
}

Save the file, restart your host, and you will see a new set of tools — browser_navigate, browser_click, browser_snapshot, browser_evaluate — available to the model.

04 · Hands-on — Playwright MCP × VWO

The first real test. We will not write a single line of TypeScript. We will write a paragraph, hand it to the LLM, and watch the browser drive itself through a real login flow against a real product.

The scenario

• Open https://app.vwo.com/ in a fresh Chromium context.
• Type an obviously invalid email and password.
• Submit the form.
• Assert that an error message appears.
• Capture a screenshot of the failure state.

The prompt — copy this verbatim in class

What the underlying tool calls look like

trace excerpt — what the LLM emitted// 1. Discover the page
browser_navigate({ url: "https://app.vwo.com/" })
browser_snapshot()  // → returns a11y tree with refs

// 2. Type credentials
browser_type({ ref: "e23", text: "[email protected]" })
browser_type({ ref: "e31", text: "WrongPassword!23" })

// 3. Submit and wait
browser_click({ ref: "e44" })
browser_wait_for({ text: "Invalid email or password" })

// 4. Evidence
browser_take_screenshot({ filename: "vwo-invalid-login.png", fullPage: true })

Notice three things students always miss on first viewing:

• The model never used page.locator('xpath=...'). It used semantic refs from the a11y snapshot — far more stable than CSS guesses.
• It chose browser_wait_for(text: ...) instead of an arbitrary sleep. Good Playwright hygiene, learned for free.
• It produced an artefact (vwo-invalid-login.png) that any human can review. Evidence first, claims second.

05 · The architecture.md blueprint

A trace from Chapter 04 is not a framework. It is a script. To get from script to framework — Pages, Modules, fixtures, environments, retry policy, reporting — we need to give the LLM something it can read and respect: an architecture.md.

Think of it as a constitution. The model reads it before every test it generates, and writes code that fits. The architecture used in this workshop is the Testing Academy 8-layer Pages + Modules framework by Pramod Dutta — a strict separation-of-concerns pattern that scales to large teams.

The 8 layers

Folder shape

The execution flow

The contract — give this to the LLM

architecture.md (excerpt)# Playwright Test Automation Framework — Architecture

## Principles
- TypeScript only. Strict mode on.
- Pages = locators only (arrow functions). No logic. No assertions.
- Modules = business logic. Use Pages. Never locate directly.
- Tests = use fixtures. Use test.step(). Use tags (@P0, @P1, @Login).
- Test data lives in src/testdata/. Never inline.
- Every test produces evidence: screenshot + trace on failure.

## Naming
- Pages:    *Page.ts    (LoginPage.ts)
- Modules:  *Module.ts  (LoginModule.ts)
- Tests:    *.spec.ts   (login.spec.ts)
- APIs:     *Api.ts     (AuthApi.ts)
- Helpers:  *Helper.ts  (WaitHelper.ts)

## Required configurations
- TS path aliases: @pages, @modules, @fixtures, @utils, @api
- Multi-browser: chromium, firefox, webkit, mobile-chrome
- Parallel execution enabled
- HTML + JSON reporting + CustomTTAReporter
- Screenshots/videos on failure, trace on first retry

Now re-prompt with the contract loaded

Before vs After — same scenario, two worlds

The generated code, in full

src/pages/LoginPage.ts — Layer 02 (locators only)import type { Page } from "@playwright/test";

export class LoginPage {
  constructor(private page: Page) {}

  // Locators as arrow functions — no logic, no assertions
  emailInput    = () => this.page.getByRole("textbox", { name: "Email" });
  passwordInput = () => this.page.getByRole("textbox", { name: "Password" });
  signInButton  = () => this.page.getByRole("button", { name: "Sign in" });
  errorMessage  = () => this.page.getByText("Invalid email or password");

  async goto() { await this.page.goto("https://app.vwo.com/"); }
  async fillEmail(v: string)    { await this.emailInput().fill(v); }
  async fillPassword(v: string) { await this.passwordInput().fill(v); }
  async submit()             { await this.signInButton().click(); }
}

src/modules/LoginModule.ts — Layer 03 (business logic)import type { Page } from "@playwright/test";
import { LoginPage } from "@pages/LoginPage";

export class LoginModule {
  constructor(
    private page: Page,
    private loginPage: LoginPage
  ) {}

  // Business logic — uses LoginPage methods, never locates directly
  async doInvalidLogin(email: string, password: string) {
    await this.loginPage.goto();
    await this.loginPage.fillEmail(email);
    await this.loginPage.fillPassword(password);
    await this.loginPage.submit();
  }
}

src/fixtures/auth.fixture.ts — Layer 06 (fixtures)import { test as base } from "@playwright/test";
import { LoginPage }   from "@pages/LoginPage";
import { LoginModule } from "@modules/LoginModule";

type Fixtures = {
  loginPage: LoginPage;
  loginModule: LoginModule;
};

export const test = base.extend<Fixtures>({
  loginPage:   async ({ page }, use) => await use(new LoginPage(page)),
  loginModule: async ({ page, loginPage }, use) => await use(new LoginModule(page, loginPage))
});
export { expect } from "@playwright/test";

src/tests/login.spec.ts — the test, six linesimport { test, expect } from "@fixtures";
import { invalidUser } from "@testdata/users";

test.describe("@P0 @Login", () => {
  test("shows error on invalid credentials", async ({ loginModule, loginPage }) => {

    await test.step("submit invalid credentials", async () => {
      await loginModule.doInvalidLogin(invalidUser.email, invalidUser.password);
    });

    await test.step("verify the inline error appears", async () => {
      await expect(loginPage.errorMessage()).toBeVisible();
    });
  });
});

A framework is a contract written in markdown. The code is just a faithful printing of it. — Chapter 05, the punchline.

Three rules students must internalise

• Pages | only locators (arrow functions) and basic UI actions. no logic no assertions
• Modules | use Page methods, never locate directly. business logic only
• Tests | use fixtures, use test.step(), use tags. @P0 @Login

06 · Custom Reporter — CustomTTAReporter

Playwright already ships excellent reporters. The point of building your own is not to compete — it is to teach students how the test runtime exposes its lifecycle. Once they have seen the events, they own the workflow.

src/utils/CustomTTAReporter.tsimport type {
  Reporter, FullConfig, Suite, TestCase, TestResult, FullResult
} from "@playwright/test/reporter";
import fs from "node:fs";

class CustomTTAReporter implements Reporter {
  private rows: any[] = [];

  onBegin(_: FullConfig, suite: Suite) {
    console.log(`▸ ${suite.allTests().length} tests queued`);
  }

  onTestEnd(test: TestCase, result: TestResult) {
    this.rows.push({
      title:    test.title,
      tags:     test.titlePath().join(" › "),
      status:   result.status,
      duration: result.duration,
      retry:    result.retry,
      errors:   result.errors.map(e => e.message),
      steps:    result.steps.map(s => ({ title: s.title, duration: s.duration })),
      attachments: result.attachments.map(a => a.path)
    });
  }

  async onEnd(result: FullResult) {
    fs.mkdirSync("tta-report", { recursive: true });
    fs.writeFileSync("tta-report/index.html", render(this.rows, result));
    fs.writeFileSync("tta-report/results.json", JSON.stringify(this.rows, null, 2));
    if (result.status === "failed") await notifySlack(this.rows);
  }
}

export default CustomTTAReporter;

playwright.config.ts (snippet)export default defineConfig({
  reporter: [
    ["list"],
    ["html", { open: "never" }],
    ["json", { outputFile: "test-results/results.json" }],
    ["./src/utils/CustomTTAReporter.ts"]
  ]
});

07 · Playwright Test Agents — Planner, Generator, Healer

Playwright now ships its own official AI agents. Three of them, each with a single job, designed to chain together. Run them independently, sequentially, or in a loop — and they will produce, run, and self-heal a complete test suite for your product.

All three are configured with one command and run inside your IDE — no separate service, no separate process. They are agent definitions shipped by Microsoft that any modern coding host (VS Code, Claude Code, OpenCode / Cursor) can pick up. Source: playwright.dev/docs/test-agents.

The three agents at a glance

One command to install the agents

Pick the host you actually use. The CLI writes the right agent definitions for that host. You can re-run this whenever Playwright updates to pick up new tools.

terminal — choose your host# For VS Code (requires v1.105+)
npx playwright init-agents --loop=vscode

# For Claude Code
npx playwright init-agents --loop=claude

# For OpenCode / Cursor and other forks
npx playwright init-agents --loop=opencode

The chain — how they hand off

The artifact layout they produce

repo layout after init-agentsrepo/
  .github/             # agent definitions Playwright wrote
  specs/               # human-readable plans (Planner output)
    basic-operations.md
    checkout-flow.md
  tests/               # generated Playwright tests (Generator output)
    seed.spec.ts       # your environment bootstrap
    basic-operations.spec.ts
    checkout-flow.spec.ts
  playwright.config.ts

The seed test — the most important file in the kit

The seed test is the contract you give the agents. It does three things at once: bootstraps the environment (logs in, seeds data, sets cookies), shows the agents your custom fixtures, and serves as the style example they imitate when generating new tests.

tests/seed.spec.tsimport { test, expect } from "./fixtures";

test("seed", async ({ page }) => {
  // Whatever a generated test will need: auth, base URL, fixtures.
  // Agents read this file to understand your conventions before they write code.
  await page.goto("https://app.vwo.com/");
  await expect(page).toHaveTitle(/VWO/);
});

An end-to-end run — invoking each agent

Once init-agents has run, the three agents appear in your IDE’s agent picker. The typical session looks like this:

Test Agents vs MCP — when to reach for which

08 · Playwright CLI + CodeGen

The third leg of the stool — and quietly, the most cost-effective. If MCP is the LLM driving the browser, the Playwright CLI is the original surface: a fast, scriptable, stateless toolkit that runs locally with zero token cost. Pair it with the LLM, and you get the cheapest path from idea to shipped test.

What is the Playwright CLI?

The CLI is the npx playwright command family — everything that ships in the @playwright/test package. Each command does one thing and exits. There is no server, no daemon, no LLM. You hand it arguments, it does the work.

It is how 90% of Playwright work happened before MCP existed. And it is still the most token-efficient way to do most tasks. The CLI is what your CI runs every hour. MCP is what you reach for when you need exploration.

The three surfaces, ranked by cost

The headline — token efficiency, MCP vs CLI

Same task. Same end result. Two very different bills. For the invalid-login VWO scenario from Chapter 04:

The 43 commands — full cheat sheet

Every Playwright CLI command and flag a working tester needs, grouped by purpose. Print this and pin it.

The Playwright extension for VS Code

Two minutes to install, ten years of saved time. The extension turns VS Code into a Playwright control room — no terminal needed for most workflows.

What you get the moment it is installed:

• Test explorer | click any test or step to run it in isolation.
• Pick locator | hover any element in the live page; the extension generates the best Playwright selector.
• Record new test | codegen, in-IDE, no terminal needed.
• Show trace | open trace files with one click.
• Inline reveal | failed assertions click straight to the source line.

Using the CLI + a prompt — the cheap path for VWO invalid login

This is the workflow you graduate to once a page is stable. Same VWO scenario as Chapter 04, but no MCP — and roughly 4× cheaper.

The LLM now has the exact selectors and the exact steps from your recording. No DOM snapshots, no tool round-trips, no browser_navigate calls. Just text in, framework-shaped code out. ~27K tokens. Done.

When to use which surface

Prompt the framework. Record the journey. Let the agents triage the failures. The keyboard becomes optional — and the bill becomes manageable. — What this whole document was leading to.

Closing — the shape of the workflow

Playwright MCP, AI Agents and CodeGen are three different surfaces of the same underlying idea: the LLM should be the author, the keyboard should be the editor, and the framework — the contract written in architecture.md — should be the constitution everything else obeys.

The companion artifacts in this kit:

• architecture.md | the contract — paste into any new repo.
• code-examples.html | the full code reference (every file written out).
• cheatsheet.html | the printable wall-poster for students.