A full curriculum from The Testing Academy: prompt engineering, LLMs, AI test design, n8n + Langflow + MCP agents, and 5 capstone projects you can ship to production.
A condensed view of the full programme. Each column maps to two modules.
Each module is taught with hands-on labs and graded mini-assignments. The full curriculum runs in cohort or self-paced mode.
Daily driver for QA prompts & brainstorming.
Long-context reasoning, code review, large transcript Q&A.
Multimodal — bug screenshots → repro steps.
Run LLaMA / Mistral / DeepSeek locally for private data.
IDE pair programming — generate Page Objects + tests inline.
Terminal agent — scaffold full Playwright frameworks from spec.
n8n is a node-based workflow tool. Drag a trigger, drag actions, connect with arrows, hit run. We use n8n to chain LLM calls, vector stores, and tools without writing glue code.
npx n8n on Node 20+. Editor at http://localhost:5678. Full walk-through: n8n tutorial.
Docker-Compose with persistent volume — same tutorial covers production setup.
OpenAI / Anthropic / Ollama chat nodes, plus a generic LangChain agent node.
The five primitives every agent workflow uses. Practise wiring each.
Cron → fetch a spec doc → LLM generates a test plan → Confluence.
Webhook on Jira → retrieval over past test cases → LLM writes new ones → comment back on Jira.
Wire these six nodes:
// Node 1 — Webhook trigger
POST /webhook/jira-issue
body: { issueKey, summary, description, ac }
// Node 2 — Vector Store retrieval (RAG)
input: $json.summary + $json.description
search: top-5 from "past-test-cases" collection
// Node 3 — Build prompt with context
system: "You are a QA engineer. Use the retrieved cases as style examples."
user: "Acceptance criteria: {{ $json.ac }}\nReturn 5 test cases as JSON."
// Node 4 — OpenAI / Claude chat node
output: $json.choices[0].message.content (parsed as JSON)
// Node 5 — HTTP Request
PUT jira/api/2/issue/{{$json.issueKey}}/comment
body: { body: "Generated test cases:\n{{ $json.testCases }}" }
// Node 6 — Slack notify
channel: #qa-agent
text: "Generated {{$json.testCases.length}} test cases for {{$json.issueKey}}"
Visual LangChain — same node-graph idea, but every node is a LangChain primitive (chain, tool, agent, memory). Great for prototyping a RAG system without code.
MCP is the open protocol that lets an LLM call your tools. Build a server, plug it into Claude / Cursor / ChatGPT, the model can read your test repo, run a script, post a comment.
Where prompt engineering ends and engineering AI systems begins. This module covers retrieval, framework code (LangChain), multi-agent crews, and how you actually evaluate & ship LLM features in CI.
ai-for-qa-part-6-crewaideepeval test run like pytest — but for prompts & agentsfrom deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric
def test_test_case_agent():
output = my_agent.run(
prompt="Generate 3 negative test cases for: login form, max 50 char password"
)
case = LLMTestCase(
input="login form, max 50 char password",
actual_output=output,
retrieval_context=["AC: password must be 8-50 chars, alphanum + symbol"]
)
assert_test(case, [
AnswerRelevancyMetric(threshold=0.8),
FaithfulnessMetric(threshold=0.85)
])
promptfooconfig.yaml — providers × prompts × tests gridname: LLM eval
on: [pull_request]
jobs:
eval:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '20' }
- name: Run promptfoo
env: { OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} }
run: npx promptfoo eval --config promptfooconfig.yaml --output report.html
- uses: actions/upload-artifact@v4
with: { name: promptfoo-report, path: report.html }
The shift from "ask ChatGPT to write a snippet" to "let an agent edit your repo, run your tests, and ship a PR." This module covers the IDE-resident agents, custom rules files, and the workflows that actually make a tester 5× faster.
CLAUDE.md, .cursorrules, AGENTS.md for QA constraints.github/copilot-instructions.md)A catalogue of agent ideas a tester can build in a weekend. Eight categories, ~50 buildable specs. Pair with the testingbuddy.ai playground for inspiration.
Turn requirements into artefacts: PRD → ACs, ACs → Gherkin, story → boundary-value test cases, user flow → exploratory charters, persona → journey tests.
Self-healing locator agent · flaky-test classifier · screenshot diff bug-detector · auto-quarantine PR opener · headless reproduction agent · failure clustering bot.
OpenAPI → REST Assured generator · contract-drift detector · mock-server generator · auth-flow agent · rate-limit explorer · GraphQL schema fuzzer.
Bug triager · duplicate detector · severity classifier · repro-step generator from a trace · screenshot-to-bug agent · sprint summary agent.
Internal docs Q&A · onboarding agent for new SDETs · runbook search · "how have we tested this before?" agent · release-notes generator from commits.
Daily test digest in Slack · trend report (LLM-summarised) · executive dashboard narrator · risk heatmap from coverage + churn · stakeholder-friendly release notes.
Hallucination detector · bias scanner · prompt-injection scanner · golden-set evaluator · structured-output validator · regression dashboard for prompts.
ATS-friendly resume rewriter · portfolio site generator · interview Q-bank from a job description · LinkedIn-headline optimiser · cold-email follow-up sequencer.
| You want to learn… | Module | Project category |
|---|---|---|
| Prompts that don't drift | AIATB_02 + AIATB_07 (PromptFoo) | Category 7 |
| n8n + Jira agent | AIATB_05 | Category 4 |
| RAG over your docs | AIATB_07 | Category 5 |
| CrewAI multi-agent | AIATB_07 | Category 1 / 2 |
| DeepEval + CI | AIATB_06 + AIATB_07 | Category 7 |
| Cursor / Claude Code skills | AIATB_08 | Category 2 |
| Copilot agent mode for testers | AIATB_08 | Category 4 |
| Vision-based bug detection | AIATB_06 | Category 4 |
One whiteboard per phase. Boxes are nodes you build; arrows are JSON or events flowing between them.
~14 weeks · 6 phases · 22 projects · 1 GitHub portfolio
Ollama → prompts → test artefacts. No cloud, no API key.
No code. Drag · connect · activate.
Docs → chunks → vectors → retrieve → LLM grounded answer.
Roles + tasks + tools. Hand-off chain ships a runnable Playwright spec.
Treat prompts like code. Eval on every PR, fail if quality drops.
Roadmap · Topic sheet · Projects → portfolio.
Every project is buildable end-to-end. Numbers match the folders in github.com/PramodDutta/AITesterBlueprint. ~14-week cohort plan; self-paced too.
Ground-floor: what an LLM is, tokens, sampling, where Ollama fits.
FastAPI + vanilla JS UI on a Llama 3.2 backend. Feed user stories, get test cases. Introduces the B.L.A.S.T. agentic protocol.
React + Monaco editor frontend, Node backend, CodeLlama brain. Migrates legacy Selenium Java to modern Playwright TypeScript.
End-to-end Java Selenium framework whose every test is generated via the RICE-POT prompt template (Role · Instructions · Context · Example · Parameters · Output · Tone).
Production-ready Markdown prompt templates that turn a PRD into context-constrained test cases. Wired into Playwright TS specs.
A full Kanban-style job tracker built entirely by Claude Code. React 19 + TS + Vite + Tailwind 4. Shows what an AI coding assistant can ship.
AI prompts + DOCX/PDF templates that rewrite a QA resume for ATS + LinkedIn. Includes follow-up email + cover-letter sequences.
Lightweight tool to grab a snapshot, annotate it, and emit a tidy bug-report Markdown ready for Jira.
Visual workflows wiring Groq, Jira REST, Google Sheets/Docs into multi-tool QA pipelines.
Schedule-driven LinkedIn post generator. n8n wakes up, asks an LLM to draft, posts on a cron.
Drag-and-drop visual flows for QA starter agents. Prompt templates, API request nodes, Groq integration.
Theory + Python walkthroughs of every RAG variant: naive, advanced, modular, agentic, graph, hybrid, fine-tuned, hyDE, self-RAG, FLARE.
Visual RAG pipelines for test-case generation. AstraDB / Chroma vector store + Groq LLM, all drag-and-drop.
Full-stack modular RAG: FastAPI + ChromaDB backend, static-HTML chat UI, domain-routed ingestion.
Teach-friendly playground: paste text, see chunks, embeddings, similarity scores. Ollama / OpenAI / Mistral providers.
Full-stack AI agent that generates test plans from Jira tickets. Express + React + TS, Groq cloud or Ollama local. A.N.T. 3-layer architecture.
Build + connect to a Model Context Protocol server. FastMCP + Playwright; expose local tools to AI assistants.
Core + advanced Python tailored to building agents. CrewAI basics + CLI patterns for orchestration.
Multi-agent crews for QA processes — autonomous task execution, memory, specialised QA agents, Jira REST integration.
Author your own MCP server exposing proprietary QA tools, dashboards, remote services as resources + prompts the AI can read.
First taste: pytest-style DeepEval tests against a local Ollama LLM. Answer Relevancy, Faithfulness, Hallucination metrics.
Three-app lab: Groq chatbot + manual metric verifier + lesson-style verifier walking through 9 metrics. FastAPI + Jinja2 + Llama 4 + GPT-4o-mini.
Three subsystems + dashboard: React e-commerce chatbot, full RAG Explorer (ChromaDB + Nomic Embed), 22-metric DeepEval framework with swappable judge LLMs (OpenAI / Groq / local Ollama).
22 projects, 6 phases, ~14 weeks. You leave with a GitHub portfolio that proves you can ship AI testing tools — not just talk about them.
— The Testing AcademyEvery project's scaffolding, n8n workflow exports, MCP server samples, prompt templates and DeepEval test suites live in one repo.
npm run start.| Folder | Contains | Run |
|---|---|---|
/01-fundamentals | LLM comparison notebooks · Ollama install scripts | Notebooks |
/02-prompts | QA prompt library, frameworks, templates (.md) | Read-only |
/03-tools | Cursor + Copilot + Claude Code config samples | Copy |
/04-generation | Test-plan, test-strategy & bug-report generators | CLI |
/05-agents | n8n exports, Langflow flows, MCP servers (TS) | npm run |
/06-advanced | Stagehand demos, ReportPortal hooks, DeepEval suites | Mixed |
/projects | The 5 capstone projects — each its own README | npm run |
Everything in this programme distils into three deliverables. Master these and you ship.
A clear 4-step path: Prompt Engineering → Generative AI → AI Agents + MCP → Advanced Tools & Integration. No detours, no hype. Open the roadmap →
8 modules — AIATB_01 → AIATB_08 — covering AI fundamentals, prompts, tool setup, AI test design, n8n + LangFlow + MCP, advanced integrations, RAG + CrewAI + DeepEval + PromptFoo, and Vibe-coding + Copilot. Open the curriculum →
Build & ship — not just read. Every student leaves with a GitHub portfolio.