AI-Native Playwright Orchestration
with BrowserBash + open-source LLMs

Teach

Open with the demo, not the theory: ask any LLM to "write a login test" for a framework it has never seen. Annotate the five defects it produces almost every time:

import { test } from '@playwright/test';          // wrong import: ignores custom fixtures
import { LoginPage } from '../pages/login.page';

test('login', async ({ page }) => {
  const loginPage = new LoginPage(page);          // manual instantiation: defeats DI
  await page.locator('.btn-primary').click();     // brittle CSS class selector
  await page.waitForTimeout(2000);                // hard wait: guaranteed flakiness
  const data: any = await page.evaluate(...);     // any: bypasses type safety entirely
});

Each line compiles. Each line is wrong for your framework. That distinction is the whole workshop.

Teach

• Generic knowledge. The model knows Playwright, the tool. It knows nothing about your fixtures, your import points, your selector policy: your architecture.
• Session amnesia. Whatever rules you explained yesterday died with that chat. Every new conversation starts from zero.
• Output inconsistency. Generic knowledge plus amnesia means every generation is a randomized coin flip against your conventions. Some days it lands. That is worse than always failing, because it teaches false trust.

The conclusion writes itself: the model does not need more intelligence, it needs your context, persistently. Give AI an architecture, not just a prompt.

Teach

• Always loaded. Every conversation starts by reading it. It is the one file that survives session amnesia.
• Never writes code. Separation of concerns: the orchestrator holds no test code, no snippets to copy. It holds rules and routes.
• Tells the AI HOW to generate. It routes: which Constitution rules apply, which skills to load, which loop to run.

Walk the real file: CLAUDE.md in the companion repo. Four blocks: Constitution, asset map, skills index, verification loop.

Teach

The payoff, line by line

Teach

step 8 loops back to step 2 until the run is green

Two anchors carry the whole loop: explore before generate (step 2) and run before report (step 8). Everything between them is bookkeeping.

The asset map

Without a map, the AI invents paths. The map's {area} placeholders are the clever part: they force the model to run ls and discover the real folder names instead of hallucinating them.

pages/{area}/[name].page.ts
test-data/factories/{area}/[name].factory.ts
fixtures/api/schemas/{area}/[name]Schema.ts
# {area} is discovered with ls, never guessed (currently: app)

Teach + demo

The routing pattern, end to end, on one prompt ("create a page object"):

Teach

Without procedure: repeat the prompt, get inconsistent output, start over, waste effort. With a skill: define the workflow once, parameterize the input, version-control the folder, get the same expert behavior every time.

Teach

• Directory: SKILL.md is required (frontmatter + instructions); optional /scripts/ for deterministic executables, /references/ for depth loaded on demand, /assets/ for templates.
• Progressive disclosure (the deck's guidance figures): level 1 is the frontmatter, roughly a hundred tokens, always loaded; level 2 is the body, kept under about five thousand tokens, loaded on trigger; level 3 is linked files, loaded only when a step needs them.
• The trigger is the most critical component. A description must answer two questions: what does this do, and when exactly should it fire.

---
name: selectors
description: >
  Chooses stable Playwright locators using the
  strict ladder. Use when writing any new
  locator, fixing a flaky one, or reviewing
  selector quality in a page object.
---

## Context
This framework bans CSS classes and XPath.
The ladder exists because role locators
survive redesigns; class names do not.

## Recipe
1. Snapshot the real element first.
2. Try getByRole(role, { name }).
3. Fall through the ladder with a comment
   for every drop.
4. Prove stability: run the spec twice.

Description craft

Teach

• Structuring for determinism: explicit numbered phases; concrete "user says X, actions, result" examples; designed error handling ("if connection refused, verify the API key"); bullets over prose, depth pushed to /references/.
• Five advanced patterns: sequential orchestration (onboard, then payment, then email); multi-MCP coordination (Figma to Drive to Linear); iterative refinement (draft, validate, loop, finalize); context-aware routing (decision trees by input); domain-specific intelligence (business rules like compliance checks before payment).
• Validation, three axes: triggering tests (fires on target phrases, stays dormant off-scope), functional tests (the calls succeed, edge cases handled), performance baselines (less manual prompting, lower total tokens).

Skills as dependencies

The companion repo pins every skill in a lockfile, exactly like package-lock.json for code. Rarely discussed anywhere, and one of the sharpest ideas in the source material: knowledge as a versioned, hash-verified dependency.

{
  "version": 1,
  "skills": {
    "selectors": {
      "source": ".agents/skills/selectors",
      "sourceType": "local",
      "computedHash": "7dd5957d812f..."
    }
  }
}

The capstone lab block

Teach

Everything so far taught the AI to write compliant Playwright. The finale closes the loop with a stack that is open source top to bottom: the AI also orchestrates and runs what it wrote, with zero API keys.

• BrowserBash (Apache-2.0): one plain-English sentence in, a real browser run out, and a machine-readable verdict back (status, extracted values, deterministic assertion results, cost). Its Verify steps compile to real Playwright checks with no model in the loop.
• Three swappable layers, free by default: provider (your local Chrome), engine (Stagehand, MIT), LLM (local Ollama first; any OpenAI-compatible server works). The default path costs nothing and phones no one.
• MCP built in: browserbash mcp exposes run_objective, run_test_file, and run_suite to any MCP host. Step 8 of the loop ("run and fix") stops being a guess and becomes a tool call with exit codes your pipeline already understands.
• qaskills.sh: a directory of 450+ ready-made QA skills installable into the same .agents/skills/ layout with one command, then governed exactly like the skills you wrote in S10: registered in the index, pinned in the lockfile.

# the run layer
npm install -g browserbash-cli
ollama pull qwen3

# plug it into the orchestrator as MCP verification tools
claude mcp add browserbash -- browserbash mcp

# install a ready-made skill from the directory
npx @qaskills/cli add playwright-e2e