Build a QA Code-Review Skill with GitHub Copilot + Playwright MCP

A reviewer that checks the app, not just the diff

Every QA team repeats the same review comments: hard-coded waits, brittle selectors, assertions that cannot fail. A skill turns those comments into a file the agent loads on demand. Adding Playwright MCP turns it from a linter into a reviewer with hands.

Two facts make this build worth doing now. First, GitHub Copilot reads the same open Agent Skills format Claude Code uses, from shared folders, so one file serves both tools. Second, Copilot code review on pull requests supports agent skills and MCP servers, now generally available on Pro, Pro+, Business, and Enterprise plans, and its MCP tool calls are read-only by design.

Check the four prerequisites

• VS Code with GitHub Copilot signed in, and agent mode available in the Chat view.
• A Copilot plan. Skills in the editor work with Copilot; the PR-review bonus in step 8 needs Pro, Pro+, Business, or Enterprise.
• Node.js 18 or newer. The Playwright MCP server runs via npx and requires it.
• A repo with Playwright tests so the skill has something real to review. Any project with tests/*.spec.ts works.

node -v
npx -v
code -v

Connect the Playwright MCP server

The server is the official @playwright/mcp package. Fastest path, one shell command that registers it for VS Code:

code --add-mcp '{"name":"playwright","command":"npx","args":["@playwright/mcp@latest"]}'

Prefer a file your whole team gets with git pull? Create .vscode/mcp.json in the repo instead:

{
  "servers": {
    "playwright": {
      "command": "npx",
      "args": ["@playwright/mcp@latest"]
    }
  }
}

• Trust prompt. The first start shows a trust dialog; approve it. You can also add servers through the Command Palette: MCP: Add Server (workspace or global), and MCP: Open User Configuration edits your user-level file.
• See the tools. In agent mode, the Configure Tools button in the chat input lists everything the server exposed: browser_navigate, browser_snapshot, browser_find, browser_click, browser_type, browser_console_messages, browser_network_requests, browser_take_screenshot, and more.
• Optional, recommended for this build: add "--caps=testing" to the args. The testing pack adds browser_generate_locator and the browser_verify_* assert-style tools, which fit a reviewer perfectly.
• Browsers install themselves. Playwright handles browser installation automatically on first use; add "--headless" if you never want a visible window, and --browser chrome|firefox|webkit|msedge to pick the engine.

Smoke-test it in agent mode with a plain prompt:

Open https://demo.playwright.dev/todomvc, take a snapshot,
and tell me the role and name of the input field.

Create the skill folder in the right place

Copilot discovers project skills from three folders, and the choice decides who else can use your skill:

For a review skill the PR bot should also run, .github/skills/ is the home. If your team also lives in Claude Code, keep a copy in .claude/skills/ (same file, two folders, still zero conversion).

Write the frontmatter (the part that decides triggering)

Only the name and description sit in the model's context permanently; the body loads when the description matches the conversation. So the description carries three jobs: what the skill does, when to use it, and the exact phrases a tester would say.

---
name: qa-code-review
description: >-
  Reviews Playwright test changes against our QA policy: locator ladder,
  wait hygiene, web-first assertions, isolation, and secret handling.
  Use when someone says "review my test changes", "QA review this diff",
  "check my Playwright tests", or a change touches tests/** or *.spec.ts.
  Live-verifies suspicious locators with Playwright MCP browser tools
  when the app is running.
---

Encode the review policy

This is your team's judgment, written once. Ours below is a sane default for Playwright + TypeScript; edit the ladder and severities to taste, but keep the two structural ideas: severity tiers (so the verdict is mechanical) and an output contract (so every review looks the same).

The locator ladder the skill enforces

getByRole  >  getByLabel  >  getByPlaceholder  >  getByText  >  getByTestId
// raw CSS, XPath, and nth() on dynamic UI are findings, not preferences

Web-first assertions, the one diff to memorize

- expect(await page.locator('.toast').isVisible()).toBe(true);
+ await expect(page.getByRole('status')).toBeVisible();

The first line checks once and moves on (race condition included, free of charge). The second retries until the element appears or times out: that retry is where most Playwright flake dies.

Teach the reviewer to verify, not guess

Add a section to the skill that tells the agent exactly when and how to use the MCP tools. The contract has one hard rule: every locator finding is VERIFIED or UNVERIFIED, and the agent never invents a result.

## Live verification with Playwright MCP
When the Playwright MCP browser tools are available AND a locator
finding exists:
1. `browser_navigate` to the page under test. Use the project's
   `baseURL` from playwright.config, or the BASE_URL env value.
2. `browser_snapshot` and check the changed locator's target exists,
   uniquely, in the accessibility tree.
3. Prefer the role and name the snapshot shows; suggest that exact
   replacement locator in the finding.
4. Tag the finding VERIFIED (say what you saw) or UNVERIFIED
   (app not running or page unreachable).
Never fabricate a verification result. In read-only contexts
(Copilot code review on PRs) inspect only: navigate and snapshot,
no clicking, no typing, no state changes.

• Why snapshots, not screenshots. browser_snapshot returns the accessibility tree as structured text, so the agent can read roles and names directly and propose getByRole('textbox', { name: 'Username' }) with evidence.
• With --caps=testing the agent can also call browser_generate_locator for a ready-made suggestion and browser_verify_element_visible to assert presence the way a test would.
• Dev server discipline. Verification needs the app running. Start it before the review, or accept UNVERIFIED tags: both are honest, silent guessing is not.

Trigger-test, then review a real (bad) diff

Test 1, triggering. Ask a natural question without naming the skill: "review my Playwright changes". If Copilot does not load qa-code-review, fix the description (add the phrase you just used) and try again. Explicit invocation always works meanwhile: /qa-code-review.

Test 2, quality. Feed it a deliberately bad spec and check every planted bug gets caught:

test('login works', async ({ page }) => {
  await page.goto('http://localhost:3000/login');   // P2: hard-coded URL
  await page.waitForTimeout(3000);                  // P1: hard wait
  await page.locator('#root > form > input:nth-child(2)').fill('admin'); // P1: brittle
  await page.locator('.btn-primary').click();       // P2: class locator
  expect(await page.locator('.toast').isVisible()).toBe(true); // P1: one-shot
});

With the dev server running, a good pass looks like this:

Commit it, and light up the PR bot

Shipping is a commit. Everyone who pulls the repo (and has step 1's prereqs: Copilot signed in, Node 18+) now has the reviewer in VS Code, the only remaining click being the one-time MCP trust prompt.

git add .github/skills/qa-code-review/SKILL.md .vscode/mcp.json
git commit -m "Add qa-code-review skill + Playwright MCP config"
git push

Then the bonus surface. Copilot code review on github.com reads the same .github/skills/ folder, and its agent-skills + MCP support is now generally available:

• Configure PR-side MCP in repository Settings → Copilot → MCP servers; auth secrets live under Settings → Secrets and variables → Agents.
• Test from a branch. Custom instructions and skills are read from the PR head branch, so you can iterate on the skill in a feature branch and watch the bot's review change before merging.
• Claude Code teammates: drop the same folder in .claude/skills/ and it fires there too, identical file.

The complete SKILL.md

---
name: qa-code-review
description: >-
  Reviews Playwright test changes against our QA policy: locator ladder,
  wait hygiene, web-first assertions, isolation, and secret handling.
  Use when someone says "review my test changes", "QA review this diff",
  "check my Playwright tests", or a change touches tests/** or *.spec.ts.
  Live-verifies suspicious locators with Playwright MCP browser tools
  when the app is running.
---

# QA code review for Playwright changes

## When to use
- A diff or PR touches tests/**, *.spec.ts, playwright.config.*, or page objects.
- Someone asks for a QA review, test review, or flake check on changed tests.

## Review workflow
1. Collect the changed test files. Review ONLY the diff, not the whole repo.
2. Walk the findings categories below in order. Note file and line for every hit.
3. If the app is reachable, live-verify every locator finding (see below).
4. Output the findings table, then the verdict, then the verified-live list.

## Findings to flag
### P0, broken safety net
- .only or .skip left on tests
- assertions inside try/catch or behind if/else that can silently pass
- missing await on expect() web-first assertions
### P1, flake sources
- page.waitForTimeout() in any form
- waitUntil: "networkidle"
- one-shot assertions: expect(await locator.isVisible()).toBe(true)
- raw CSS, XPath, or nth() locators on dynamic UI
### P2, maintainability
- locator ladder violations; prefer getByRole > getByLabel >
  getByPlaceholder > getByText > getByTestId, and anything below
  the ladder needs a comment explaining why
- copy-pasted setup instead of fixtures
- hard-coded URLs or credentials; use baseURL and env config
### P3, style
- console.log debris, dead code, vague test names

## Live verification with Playwright MCP
When the Playwright MCP browser tools are available AND a locator
finding exists:
1. browser_navigate to the page under test. Use the project's baseURL
   from playwright.config, or the BASE_URL env value.
2. browser_snapshot and check the changed locator's target exists,
   uniquely, in the accessibility tree.
3. Prefer the role and name the snapshot shows; suggest that exact
   replacement locator in the finding.
4. Tag the finding VERIFIED (say what you saw) or UNVERIFIED (app not
   running or page unreachable).
Never fabricate a verification result. In read-only contexts (Copilot
code review on PRs) inspect only: navigate and snapshot, no clicking,
no typing, no state changes.

## Output format
A findings table with columns: File:Line | Severity | Finding | Fix.
Then: Verdict: approve or request changes, with one line of reasoning
(any P0 or two or more P1 findings mean request changes).
Then: Verified live: the list of checks performed, or "none
(app not reachable)".

## Guardrails
- Review the diff you were given; do not rewrite files unless asked.
- Quote the exact offending line in every finding.
- If there are zero findings, say so, and still report what you verified.
- The output is advice for a human reviewer, not an auto-merge signal.

Troubleshooting