The Testing Academy · Masterclass · QA Engineering

Loop Engineering for QA.

Design QA systems that observe evidence, make one bounded change, verify independently, preserve the record, and stop for a human decision.

3 parts 18 lessons Playwright project Evidence + guardrails

Before you begin

Bring a safe, disposable target.

The included project supplies the broken test, protected product fixture, immutable oracle, and deterministic maker. No AI API key or production access is required.

Open the setup lesson

Runtime: Node.js 20 or newer, npm, and Git.
Browser: Chromium installed through Playwright.
Working knowledge: Read a locator failure, a web-first assertion, and a Git diff.
Safety boundary: Use only the synthetic fixture. Keep credentials and production systems outside the loop.

Part one · Foundations

Design the Control System

Separate a useful engineering loop from a long prompt, a retry, or an unconstrained agent. Define what the system may observe, change, prove, and escalate.

1.1

Loop Engineering in QA

The feedback model, the QA fit, and the authority boundary that keeps an agent from grading itself.

1.2

Prompt, Context, Harness, Loop

Classify each layer by the engineering question it answers and the failure it can prevent.

1.3

Goal and Observable Exit

Turn a vague request into a measurable result with explicit acceptance and escalation states.

1.4

Evidence Before Action

Choose the smallest trustworthy evidence bundle: failure output, source, oracle, and scope.

1.5

The Bounded Maker

Give the repair worker one narrow correction surface and no authority over acceptance.

1.6

State and Stop Decisions

Record attempts outside the conversation and make iterate, stop, and escalate explicit.

The control map

A loop is more than a retry.

A retry repeats an operation. A trustworthy loop uses verified evidence to choose the next bounded action, records what happened, and stops when authority must return to a human.

Excalidraw diagram of the six-part QA control loop from goal to human approval — GOAL → OBSERVE → BOUNDED ACTION → VERIFY → RECORD → DECIDE Editable Excalidraw source

Part two · Trust

Build Trustworthy Verification

Protect product intent and test strength with a machine-readable contract, immutable acceptance checks, tamper-evident evidence, and an independent reviewer.

2.1

Machine-Readable Contract

Encode the goal, paths, checks, attempt ceiling, and final state in structured data.

2.2

Mutable and Protected Paths

Separate the correction surface from product code, oracle, policy, prompts, and controller.

2.3

Deterministic Verifier

Use fixed commands and exit codes instead of a model's narrative confidence.

2.4

Immutable Oracle

Preserve intended behavior separately so a weak candidate cannot rewrite acceptance.

2.5

Evidence Hashes and Manifests

Make every pass claim traceable to command output and a stable evidence identity.

2.6

Independent Review and Human Gate

Challenge false-green results, then stop at a decision a person must own.

Verified in Chromium

The tutorial begins red and earns green.

The included project uses a synthetic login fixture. The original test asks for an obsolete accessible name. The maker may edit only the test, while the product and oracle remain sealed.

01PolicyPASS

01Target testFAIL

01Immutable oraclePASS

02PolicyPASS

02Repeated target testPASS

02Immutable oraclePASS

✓Controller stateAWAITING_HUMAN_APPROVAL

Synthetic login fixture used by the bounded Playwright repair loop — IMMUTABLE PRODUCT FIXTURE · SYNTHETIC DATA · NO PRODUCTION ACCESS

Part three · Project

Run the QA Repair Loop

Execute a complete Playwright example, inspect both failed and accepted evidence, and adapt the pattern to test design, STLC, Selenium, and AI evaluation.

3.1

Build the loop. Preserve the evidence. Keep the final decision human.

Open all lessons