Seven AI Skills
Every QA Should Actually Learn

A prompt is what you type, a skill is what you save, an AI agent is the system that uses both. Knowing which rung you are on decides whether your AI testing work is a one-off or an asset your whole team reuses.

The ladder: prompt -> skill -> agent

• Prompt: typed every time. Instructions that live in your head or a notes file, with zero reuse guarantee.
• Vibe-coded prompt: iterated, still disposable. A longer prompt you refine until it works, but it is still typed per session and dies with the chat.
• Skill: a folder the agent loads on demand. A SKILL.md with YAML frontmatter (name plus description) and Markdown instructions: reusable, shareable, version controlled.
• AI agent: the loop that uses both. A system that plans, calls tools, observes results, and loops until the goal is met.
• Each rung reuses the one below. A skill packages a proven prompt; an agent loads skills while it works: that is the whole ladder.

Promote a prompt into a skill

1. Catch the repeat. The third time you type the same test-case prompt, stop: anything retyped is a skill candidate.
2. Create the folder. Make a directory with a SKILL.md inside: YAML frontmatter on top, instructions below.
3. Write the description for matching. The agent loads a skill only when the description matches the request, so name the exact situations it should fire in.
4. Move the prompt body into the Markdown section. Your proven wording becomes the instructions the agent follows.
5. Commit it to git. Now it is version controlled: reviewable in a pull request and shareable with the whole team.
6. Let the agent drive. On a matching request the agent plans, calls tools, observes results, and loops until the goal is met, with your skill guiding it.

---
name: qa-test-case-writer
description: Generate test cases from a user story or requirement.
  Use when asked to write, expand, or review test cases.
---

Write test cases in Given/When/Then form.
Always cover happy path, negative, and boundary cases.

That is the short version: for worked examples of all three rungs, read the full breakdown, then start your own library with the skill masterclass.

RAG stops you from hoping the model memorised your product: it retrieves chunks from your own documents and pastes them into the prompt as context. For a tester, that is the difference between a test case you can trace to a spec and a confident guess you cannot.

What RAG actually is

• RAG in one line. Retrieval Augmented Generation: instead of trusting whatever the model memorised in training, you retrieve the most relevant chunks from your own documents and hand them to the model as context for its answer.
• Embeddings are the engine. An embedding turns text into a vector of numbers so that similar meanings sit close together. Retrieval is then a similarity search over those vectors: matching by meaning, not by keyword.
• Chunking is a tuning knob. Documents are split into pieces before embedding. Too big and retrieval gets noisy, too small and the context is lost. Getting this wrong is the most common reason a RAG setup feels dumb.
• Grounding makes hallucinations checkable. If an answer cites no retrieved chunk, it is unsupported. That one rule turns "sounds right" into something you can actually verify.
• The QA payoff. Ground test generation in your real specs, past bug reports, and requirements, so every generated case traces back to a real source instead of a guess.

Build your first QA RAG, step by step

1. Collect your sources. Gather the documents that define what correct looks like: specs, requirements, past bug reports. This is the ingest stage.
2. Chunk them. Split each document into passages: small enough to be specific, big enough to keep their meaning intact.
3. Embed every chunk. Run each chunk through an embedding model to get a vector of numbers. Similar meanings land close together in that vector space.
4. Store the vectors. Load them into a vector database so they can be searched by similarity later.
5. Ask and retrieve. Embed your question the same way, then retrieve the top matching chunks for it.
6. Generate with receipts. Paste the retrieved chunks into the prompt and instruct the model to answer only from them, citing which chunk supports each claim.

SPEC CHUNK (retrieved from checkout-spec, section 4.2):
"An invalid promo code shows error PROMO-401 and
leaves the cart total unchanged."

QA QUESTION:
"What should happen when a user applies an invalid promo code?"

GROUNDED ANSWER:
Show error PROMO-401 and keep the cart total unchanged.
Source: checkout-spec, section 4.2.

DERIVED TEST CASE:
1. Add any item to the cart
2. Apply an invalid promo code
3. Assert error PROMO-401 is displayed
4. Assert the cart total did not change

Go deeper with the QA-focused walkthrough at /ai/rag-tutorial-for-qa, then level up with the advanced build (Langflow, BGE-M3, Chroma, reranking) at /masterclass/advanced-rag.

The Playwright Agent CLI puts a real browser at the end of your terminal, built so an AI agent can drive it without drowning in raw HTML. Sessions, network inspection, route mocking, and tracing all live behind one binary: playwright-cli.

Why agents love this CLI

• What it is. A command line front end to a real browser: open pages, inspect network traffic, mock or block requests, and record traces without writing a test file first.
• Token-efficient by design. An agent does not need the whole page, it needs just enough structure to act. The CLI is built around that idea, which keeps agent runs cheap and fast.
• Ref-based snapshots. Instead of dumping raw HTML, the CLI returns a snapshot where each element carries a ref. The agent acts on the ref directly and never reads the whole DOM.
• Named isolated sessions. A flag like -s=cart gives a flow its own isolated session, so your cart experiment and your login experiment never share state.
• Skills for your agent. One command, playwright-cli install --skills, installs the skill set an agent can call to drive the browser on its own.

Hands-on: your first session

1. Install once. Run npm install -g @playwright/cli. This provides the playwright-cli binary.
2. Check your setup. Print the resolved config with playwright-cli config so surprises surface now, not mid-run.
3. Open a named session. Start a headed, isolated session against the practice cart: playwright-cli -s=cart open https://app.thetestingacademy.com/playwright/ttacart/ --headed.
4. Watch the wire. Inspect what the page is calling with playwright-cli network, then mock or block the API layer with playwright-cli route "**/api/**".
5. Record the evidence. Run playwright-cli tracing-start, drive the flow, then save it with playwright-cli tracing-stop trace.zip.
6. Review and extend. Open the dashboard with playwright-cli show, and hand your agent the toolkit with playwright-cli install --skills.

npm install -g @playwright/cli
playwright-cli config
playwright-cli -s=cart open https://app.thetestingacademy.com/playwright/ttacart/ --headed
playwright-cli network
playwright-cli route "**/api/**"
playwright-cli tracing-start
# drive the flow in the browser
playwright-cli tracing-stop trace.zip
playwright-cli show
playwright-cli install --skills

Every command above comes straight from the full walkthrough at /masterclass/playwright-cli: start there when you are ready to wire the CLI into a real agent workflow.

Your product just shipped a feature that writes its own output. assertEquals cannot grade a paragraph, so if you want to be the tester who signs off AI features, you need a new kind of assertion: the metric.

Why assertEquals cannot grade a paragraph

• Same input, different valid outputs. An LLM feature is non-deterministic by design. There is no single expected string, so there is nothing for assertEquals to pin.
• String equality grades characters, not meaning. Two answers can share zero words and both be right, or match closely and be subtly wrong. Equality fails the first and passes the second: the exact opposite of what you want.
• A metric is the replacement. It measures one property of the output (relevancy, faithfulness, toxicity) and returns a score from 0 to 1. Evaluation replaces equality with measurement.
• The threshold is yours. You compare the score against a threshold you own, and that comparison is a deterministic pass or fail your pipeline understands.
• Direction matters. Quality metrics treat the threshold as a floor: higher is better. Safety metrics treat it as a ceiling: lower is better. Confuse the two and toxic output passes.

The judge loop: your first eval

1. Capture a real test case. Save the input, the retrieved context if the feature uses RAG, and the exact output the model produced.
2. Pick one metric and its direction. Quality metrics like answer relevancy or faithfulness need a floor; safety metrics like hallucination or PII leakage need a ceiling.
3. Choose a judge that is not the model under test. LLM-as-judge means a second, trusted model does the grading. The grader and the graded are never the same model, no exceptions.
4. Score and collect the reason. The judge returns a score from 0 to 1 plus a written reason, so a failure tells you why it failed, not just that it failed.
5. Assert against your threshold. DeepEval is pytest-native, so a metric that misses its threshold fails the build like any red test. PromptFoo and TruLens are the other two tools to know.
6. Rerun on every change. New prompt, new model, new context source: run the suite again. That is regression testing for a feature that writes its own output.

Evaluation is the only way to regression-test a feature that grades differently every run, and the tester who owns the metrics and thresholds is the one who signs off the AI feature. The full pytest setup, metric by metric, is in the DeepEval masterclass.

Cursor is an AI-first code editor, and for QA its value is bounded, reviewable agent work on your test repo. The discipline is short: rules first, one small task, deterministic verification before you accept a single diff.

The workflow: constrain before you trust

• Treat the agent like a fast junior. It is quick and confident, and it needs what a new hire needs: written conventions, a small first ticket, and a reviewer who actually runs the code.
• Privacy-aware setup comes first. Decide what the editor may see and send before pointing it at a work repo. Nothing else matters if that answer is wrong.
• Rules beat prompts. A prompt steers one task, a project rule constrains every task: locator policy, naming, paths the agent must not touch, written once.
• Small scope is the safety feature. One spec, one page object, one fixture per task. Small tasks produce small diffs, and small diffs are the only ones you can genuinely review.
• Deterministic verification is the gate. The agent's summary is not evidence. The suite passing is. Run it before you accept anything.

One bounded task, end to end

1. Write project rules. Capture repo conventions as rules the agent must follow: selector strategy, folder layout, files that are off limits.
2. Scope one small task. "Add a spec for the empty cart state" beats "improve coverage". If you cannot predict the shape of the diff, shrink the task.
3. Pick the least-autonomous agent mode that works. Bounded agent modes exist so you choose the leash length: the goal is a proposed diff you review, not silent edits.
4. Feed context deliberately. Use MCP when the task needs tools or data beyond the repo, and target your Playwright assets so the output lands in code your suite can check.
5. Verify before accepting. Read the diff, then run the suite. Green is the acceptance gate. Red means reject or re-scope, never "accept and fix later".
6. Add the second net. Use CLI review to inspect the work, let Bugbot run an automated review pass, and fold any repeated mistake into a new rule.

The full setup, from privacy-aware configuration to Bugbot, is in the Cursor masterclass, with the field lessons collected in the follow-up.

A skill directory is your personal library of SKILL.md folders: versioned in git, discovered by your agent, loaded only when a request matches. Prompts scattered across notes files die with the session; a directory gets reviewed, shared, and pulled like code.

A versioned library, not a pile of prompts

• One skill, one folder. Each skill is a folder holding a SKILL.md: YAML frontmatter with a name and a description, then Markdown instructions. The directory is simply your collection of those folders under version control.
• Progressive disclosure keeps it cheap. Only the name and description sit in context at all times, about 100 tokens. The body loads when a request matches the description, and referenced files load only when a step needs them. That is why dozens of skills stay cheap.
• Reuse across projects. A prompt you retype is a prompt you mutate. A skill in ~/.claude/skills/ follows you into every repo, identical every time.
• Review in pull requests. A SKILL.md change ships through the same PR flow as code: diffed, commented, approved. A prompt in a notes file gets none of that.
• Distribution by git pull. Commit the folder once and every teammate inherits the whole library on their next pull. No pasted prompts in chat, no "which version do you have".
• Portability across agents. Claude Code, GitHub Copilot, and Hermes all discover skills from folders, and VS Code Copilot reads .claude/skills/ too, so one repo folder can serve two agents.

Build and ship your first skill

1. Create the folder. In your test repo run mkdir -p .claude/skills/flaky-test-triage: one folder per skill, named after the job it does.
2. Write the frontmatter. Add a SKILL.md whose description carries the exact trigger phrases teammates type, like "flaky test" or "quarantine this spec". The agent matches requests against this text, so a vague description never fires.
3. Keep the body small. Below the frontmatter, write short Markdown steps with exact commands and expected output. The body only loads on a match, and a tight body keeps the loaded context sharp.
4. Split out heavy parts only when needed. Move long checklists into references/ and helper code into scripts/. They load only when a step points at them.
5. Commit it. Run git add .claude/skills/ && git commit: the skill now has history, blame, and a reviewable diff.
6. Let the team pull it. Teammates run git pull and their agent discovers the new skill automatically. Shipping a capability becomes a one-line changelog entry.

.claude/skills/
└── flaky-test-triage/
    ├── SKILL.md
    ├── references/
    │   └── quarantine-checklist.md
    └── scripts/
        └── rerun-failed.sh

# SKILL.md
---
name: flaky-test-triage
description: Triage a flaky Playwright test. Use when the user
  says "flaky test", "quarantine this spec", "passes on retry",
  or pastes an intermittent CI failure.
---

## Steps
1. Re-run the failing spec alone, then with the full suite.
2. Compare the failing trace against the last green run.
3. Quarantine with an owner and a ticket, never silently.

Do not start from zero: our ready-made 36-skill QA suite covers the STLC end to end and drops straight into your directory. Grab it at /masterclass/skill-masterclass.