LIVE AI Tester Blueprint New batch | ₹35,000₹9,99933% OFF Code AITESTER Join
QA AI Agents  /  Bug Triage Agent
Langflow · latest
The Testing Academy · Masterclass 2026

QA AI Agents,
built node by node.

A hands-on series where we build production-style QA agents in Langflow — no glue code. Each agent reads real engineering data, reasons over it with an LLM, and returns strict JSON your tools can consume.

3
AI Agents
7–9
Nodes / flow
1
Shared stack
JSON
Every output
i
How to use this guide

One pattern, three agents

Every agent in this series follows the same backbone: get the data → make it readable → tell the model what to decide → let it reason → force the shape → ship the result. Learn it once on Agent 1, and Agents 2 and 3 are variations on the same theme.

AgentReadsDecidesStatus
Bug TriageA Jira ticketSeverity, priority, impact, RCALive
Flaky Test AnalyzerTwo Playwright runsFlaky vs real failure, rerunsLive
API Contract ValidatorYour cURL requestsStatus + schema conformance, pass/failLive
Teach
Open with the backbone on a whiteboard before touching Langflow. Students who see the six-step shape first stop treating each node as magic and start reading the flow as a sentence.
Langflow 101 · Foundations

Langflow basics,
from zero to Hello Groq.

Before we build the three QA agents, get comfortable with Langflow itself: what it is, how to install it, the components you'll wire together, and two starter flows — a plain Hello World and a Hello Groq that actually calls an LLM.

~2 min
To install
6
Core component types
2
Starter flows
OSS
Free & local
1
Foundational · the idea

What is Langflow?

Langflow is an open-source, visual builder for LLM and agent apps. You drag components (nodes) onto a canvas and connect their ports; each wire carries a typed value (a Message, Data, or a LanguageModel). No glue code — the flow is the program.

What it gives youWhy it matters for QA
Visual drag-and-drop canvasSee the whole pipeline as a sentence: input → reason → output
Built-in PlaygroundRun and test a flow instantly, no deploy
Every flow is also an API endpointCall your agent from CI, a script, or another tool
Import / export as JSONShare a flow with students in one file (that's what our 3 agents are)
Python under the hoodDrop into a custom component when a node doesn't exist
Teach
Frame Langflow as "Scratch for LLM apps." Beginners stop fearing the model when they see it's just one node on a wire between an input and an output.
2
Setup

How to install Langflow

Python 3.10–3.13. Use a fresh virtual environment so Langflow's dependencies stay isolated. Pick one of the three ways below, then open http://127.0.0.1:7860.

terminalbash
# 1) fresh virtual env (recommended)
python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate

# 2) install Langflow
pip install langflow

# 3) start it (opens the canvas on :7860)
langflow run
terminalbash
# uv is a fast Python package manager
# install: https://docs.astral.sh/uv/

# run Langflow without a manual venv
uvx langflow run

# or install into a project
uv pip install langflow
langflow run
terminalbash
# no Python setup needed - just Docker
docker run -it -p 7860:7860 langflowai/langflow:latest

# then open http://127.0.0.1:7860
First launch is slow. Langflow downloads starter projects and builds its component index on the first run. Give it a minute, then refresh :7860.
3
The palette

Core components of Langflow

Components are grouped in the left palette. You'll build almost everything in this masterclass from these six families. Drag one out, and its input/output ports tell you what it accepts and emits.

1 · Inputs & Outputs

Chat InputWhere the user/operator types. Emits a Message.
Chat OutputRenders the final answer in the Playground.
Text Input / OutputPlain text in/out for non-chat flows.

2 · Prompts

PromptA template with {variables} that get filled from wires.
System / instructionsTell the model what role and format to use.

3 · Models (LLMs)

GroqFast inference. The model we use across all 3 agents.
OpenAI / Anthropic / GoogleSwap-in alternatives, same Message port.
OllamaRun a local model, no API key.

4 · Data & Loaders

FileLoad a CSV / JSON / PDF into the flow.
API RequestCall a REST endpoint (e.g. Jira). Emits Data.
URL / DirectoryPull content from a page or folder.

5 · Processing & Helpers

Parse DataFlatten Data → readable Message for the model.
Structured OutputForce the answer into a fixed JSON schema.
Split / Combine TextChunk or merge before/after the model.

6 · Agents, Memory & Vector Stores

AgentTool-calling LLM that decides which tool to run.
Chat MemoryRemembers prior turns in a conversation.
Vector StoreChroma / Qdrant / Astra for RAG retrieval.
The wire colour is the data type. A Message port only connects to a Message port, Data to Data, and so on. If two ports won't connect, the types don't match — that's the type system protecting you, not a bug.
4
Starter flow · 01

Hello World — your first flow

The smallest possible flow: take input, echo it back. Two nodes, one wire. This proves Langflow runs and teaches you how to connect ports.

Langflow canvas
IN
Chat Input
Where you type

Drag it from Inputs. It emits whatever you type as a Message.

out: Message
OUT
Chat Output
What you see

Drag it from Outputs. Connect Chat Input's Message port into it.

in: Message

Build it

New Flow → Blank Flow. Drag Chat Input and Chat Output onto the canvas. Drag a wire from the Input's output dot to the Output's input dot.

Run it

Click Playground (top right). Type hello world and send. The Output echoes it back. That's a working flow.

Why start this small. Once a student connects two ports and sees data move, every bigger flow is just more nodes on the same kind of wire.
5
Starter flow · 02

Hello Groq — call a real LLM

Now put a model in the middle. Add the Groq component between input and output, paste a free API key, and your flow starts thinking.

Langflow canvas
IN
Chat Input
Your question

e.g. "Explain flaky tests in one line."

out: Message
AI
Groq Model
The reasoning step

From Models. Paste your GROQ_API_KEY and pick a model. Wire the input Message into its Input.

in: Messageout: Message
OUT
Chat Output
The model's reply

Connect Groq's Message output here.

in: Message

Get a free Groq key

Sign in at console.groq.comAPI Keys → Create. Copy it once (you can't see it again).

Add & wire the Groq node

Drag Groq between Chat Input and Chat Output. Paste the key into Groq API Key. Pick a model such as llama-3.3-70b-versatile or openai/gpt-oss-120b.

Run it

Open the Playground, ask your question, and Groq answers. You just built an LLM app with three nodes.

.envkeep keys out of the flow
# Better than pasting the key into the node:
# set it as an environment variable / Langflow global variable

export GROQ_API_KEY="gsk_your_key_here"

# then reference it in the Groq component's API key field
# as a global variable -> export the flow WITHOUT the secret
Never commit the key. Store GROQ_API_KEY as a Langflow global variable and export flows without it. The 3 agent JSONs on this site ship with empty secret fields for exactly this reason.
6
The pattern

The overall flow — one shape, every agent

Hello Groq is the seed. Add a Prompt to set the instructions and a Structured Output to force JSON, and you have the exact backbone all three QA agents use.

Langflow canvas - the universal backbone
IN
Input / Data
Get the raw material

Chat Input, an API Request (Jira), or a File (Playwright report).

out: Message / Data
P
Prompt
Tell the model what to decide

The instructions + the fields you want back.

in: varsout: Message
AI
Groq Model
Reason over it

Low temperature so verdicts stay stable.

in: promptout: Message
[ ]
Structured Output
Force the shape

Re-extract the answer against a fixed JSON schema.

in: text + llmout: Data
OUT
Output
Ship it

Show JSON, or write back to Jira / Slack / CI.

in: Data
Teach
Say it out loud: get the data → tell the model what to decide → let it reason → force the shape → ship the result. Every agent in the next three lessons is this sentence with different inputs.

Ready? Jump into Agent 01 · Bug Triage and watch this backbone come alive on a real Jira ticket.

Agent 01 · Triage

Bug Triage AI Agent

Hand it one Jira issue key. It fetches the ticket, reads it with Groq, and returns severity, priority, impact areas, and a root-cause hypothesis — as strict JSON.

7
Nodes
5
JSON fields
1
Manual input
120B
gpt-oss model
1
Foundational · the problem

What we're solving

Objective

A QA lead opens a fresh Jira ticket and decides by hand: how bad is this, how fast must it be fixed, what does it touch, and what likely caused it. Across a backlog that's hours of repetitive judgement — and it drifts between people.

We want to hand the agent one issue key, have it fetch the ticket, reason over the description, and return a consistent verdict in a machine-readable shape — ready to feed dashboards, auto-label tickets, or route to a team.

Teach
Ask the room: "what decision are we automating, and what does good output look like?" The answer — severity + priority + impact + RCA, as JSON — dictates every node we add. Architecture follows the contract.

Five required outputs map to a straight-line pipeline. Each node has one job; the chip colour is the data type on the wire.

Langflow canvas
IN
Chat Input
Jira issue key

Operator types one key, e.g. PROJ-123. The only manual input.

out: Message
API
API Request
Fetch the ticket from Jira

Calls Jira Cloud REST v3 /issue/{key} with Basic auth; returns summary, description, priority, components, labels.

in: keyout: Data
{ }
Parse Data
Flatten JSON to text

Maps nested Jira fields into a clean text block. Models reason better over prose than deep JSON.

in: Dataout: Message
P
Prompt Template
Triage instructions

Wraps the ticket with the triage brief and the five fields to decide. {issue} is filled by the node above.

in: issueout: Message
AI
Groq Model
The reasoning step — gpt-oss-120b

Runs the triage thinking on Groq's OpenAI-compatible endpoint. Temperature 0.1 so verdicts stay stable.

in: promptout: Messageout: LanguageModel
[ ]
Structured Output
Force the answer into JSON

Takes the prose and a LanguageModel handle, re-extracts against a fixed schema. Free text becomes reliable fields.

in: textin: llmout: Data
OUT
Chat Output
Show the verdict

Renders the JSON in the Playground. In production, swap for a node that writes back to Jira or posts to Slack.

in: Data
Why two wires leave the Groq node. Structured Output needs both the text to extract from and a live model to do the extracting. Groq exposes a Message output (the answer) and a LanguageModel output (the model). Both feed the same node. Point this out — it trips up most beginners.

Severity and priority are different axes. Severity = how broken. Priority = how urgently we fix it. The model picks from a closed set on each, so output stays consistent.

Severity — technical impact

BlockerUnusable / data loss / no workaround
CriticalCore feature down, painful workaround
MajorSignificant function impaired
MinorSmall defect, easy workaround
TrivialCosmetic, typo, polish

Priority — business urgency

P0Drop everything, fix now
P1This sprint, top of queue
P2Planned, near-term
P3Backlog, when capacity allows
P4Nice to have / maybe never
Teach
The exam question: a typo in the CEO's name on the landing page — low severity (cosmetic), high priority (P0/P1, public and embarrassing). The two axes diverging is the whole reason we capture both.

This text lives inside the Prompt Template node — the entire reasoning spec. {issue} is replaced at runtime with the parsed ticket.

triage_prompt.txtprompt
You are a senior bug triage engineer. Analyze the Jira issue below and produce a structured triage.

JIRA ISSUE:
{issue}

Assess and decide ALL of the following:
1. SEVERITY - technical impact. One of: Blocker, Critical, Major, Minor, Trivial.
2. PRIORITY - business urgency. One of: P0, P1, P2, P3, P4.
3. IMPACT_AREAS - modules, journeys, or systems affected.
4. ROOT_CAUSE_ANALYSIS - best hypothesis of the underlying cause.
5. JUSTIFICATION - one or two sentences on the severity/priority call.

Be decisive. Base every conclusion only on the issue content. Do not invent stack traces or logs.
Design choice. The prompt does the reasoning; Structured Output does the formatting. Splitting the two means you can tighten the schema without touching the reasoning, and vice-versa.

Import the JSON, or rebuild from scratch. Either way you only fill two secrets.

Get the flow into Langflow

Download the JSON below. In Langflow: project → New FlowImport → pick the file. All seven nodes appear pre-wired.

Create a Jira API token

At id.atlassian.com → Security → API tokens. Base64-encode email:token and put it in the API Request Authorization header as Basic <value>.

Point the URL at your Jira

Replace YOUR_DOMAIN in the API Request URL with your site. Keep the ?fields=… query — it keeps the response small.

Add your Groq key

Get a key at console.groq.com (gsk_…). Paste into the Groq node. Model is preset to openai/gpt-oss-120b.

Run it

Open Playground, type PROJ-123, send. Watch the data move node to node; the triage JSON lands in chat.

Don't commit secrets. When you export to share with students, store the Groq key and Jira token as Langflow global variables and export without keys. The download below ships with empty secret fields.
Download bug_triage_agent.json

Test the agent like you'd test any classifier — on the edges where its judgement is hardest.

T01

Severity ≠ priority: feed a cosmetic typo on a public page. Verify it returns low severity but high priority — not both low.

T02

Sparse ticket: a one-line description with no repro. The RCA field should say what's unknown, not invent a stack trace.

T03

Schema conformance: run 10 tickets. Every output must contain all five keys with valid enum values for severity/priority.

T04

Auth failure: use a bad Jira token. Verify the flow surfaces the API error instead of triaging an empty body.

T05

Determinism: same ticket, 5 runs. At temp 0.1 the severity/priority verdict should not drift.

2
The output contract

What a run returns

{
  "severity": "Critical",
  "priority": "P1",
  "impact_areas": ["Checkout", "Payment gateway", "Order email"],
  "root_cause_analysis": "Null order ID reaching the payment callback — likely a race between cart-clear and webhook receipt.",
  "justification": "Revenue path broken for a subset of users with a fragile workaround, so Critical severity and P1 urgency."
}
Agent 02 · Reliability

Flaky Test Analyzer

Give it two Playwright runs of the same suite. It separates genuinely flaky tests from real, reproducible bugs — and tells you exactly which ones to rerun.

2
Runs compared
3
Verdict classes
1
Custom node
JSON
Playwright report
1
Reliability · the problem

Flaky, or actually broken?

Objective

A suite goes red. Half the failures are real bugs; the other half are flaky tests that pass on the next run. Teams waste hours re-running everything and lose trust in the suite. The two need opposite responses: bugs go to engineering, flakes get quarantined or rerun.

We hand the agent two Playwright JSON reports — Build 1 and Build 2 of the same suite — and it tells us which tests are flaky, which are consistently failing, and what to do with each.

The definition that matters. A test that fails in both runs is not flaky — it's a reproducible bug. True flakiness is non-deterministic: pass in one run, fail in the other, or a pass that only happened after a retry. This agent encodes that distinction so students stop mislabelling bugs as "flaky."
Teach
This is the highest-value lesson in the series. Most engineers call any red test "flaky." Show that flakiness is about variance across identical runs, not about whether a test failed.
Langflow canvas

Two file inputs feed a custom comparison node, then the same reason → shape → ship backbone as Agent 1.

F1
File
Build 1 — results.json

The Playwright JSON report from the first run. Carries status, retries, ok per test.

out: Data
F2
File
Build 2 — results.json

The JSON report from the second run of the same suite.

out: Data
Custom Component
Flaky Diff — compare two runs

Indexes both reports by test name, then classifies each: flaky (mixed / retry), consistent failure (both), or stable. The deterministic core — done in code, not by the LLM.

in: run1in: run2out: Message
P
Prompt Template
Reliability brief

Gives the model the strict flaky-vs-bug definitions and asks for per-test cause hypotheses + rerun advice.

in: comparisonout: Message
AI
Groq Model
gpt-oss-120b

Adds the reasoning the diff can't: why each test is flaky, likely root causes for the real failures.

in: promptout: Messageout: LanguageModel
[ ]
Structured Output
Force into JSON

Schema: flaky_tests, consistent_failures, rerun_recommendation, counts, summary.

in: textin: llmout: Data
OUT
Chat Output
Show the report

In production, post to Slack or open Jira tickets for the consistent failures automatically.

in: Data
Why a custom component here. The classification (mixed vs both vs retry) is deterministic logic — it must be exact and free. We do it in code, then hand the clean comparison to the LLM only for the parts that need judgement: cause hypotheses and recommendations. Don't make the model count.

The heart of the agent. This runs inside the Flaky Diff custom component — pure Python, deterministic, no LLM.

flaky_diff.pypython
def build_comparison(self):
    r1 = self._index(self.run1)   # {test_title: {status, retries}}
    r2 = self._index(self.run2)
    all_tests = sorted(set(r1) | set(r2))

    flaky, consistent_fail, retry_flaky, stable = [], [], [], []
    for t in all_tests:
        a  = r1.get(t, {}).get("status", "absent")
        b  = r2.get(t, {}).get("status", "absent")
        ra = r1.get(t, {}).get("retries", 0)
        rb = r2.get(t, {}).get("retries", 0)

        if {a, b} == {"passed", "failed"}:           # mixed across runs
            flaky.append(t)                          # -> FLAKY
        elif a == "failed" and b == "failed":        # failed in both
            consistent_fail.append(t)                # -> REAL BUG
        elif a == b == "passed" and (ra or rb):      # passed, but retried
            retry_flaky.append(t)                    # -> FLAKY
        elif a == b == "passed":
            stable.append(t)                         # -> STABLE
    return report(flaky, retry_flaky, consistent_fail, stable)
Build 1Build 2VerdictAction
passfailFlaky (mixed)Rerun / quarantine
failpassFlaky (mixed)Rerun / quarantine
pass (retry)passFlaky (retry)Investigate wait/timing
failfailConsistent failureFile as a bug
passpassStableNone
Teach
Run the component on the two sample reports live. Seeing checkout (failed both) land in consistent_failures while profile (pass→fail) lands in flaky makes the definition click instantly.

The reliability brief inside the Prompt Template node. The deterministic diff has already classified everything — the model adds the why.

reliability_prompt.txtprompt
You are a senior test reliability engineer. You are given a comparison of two Playwright runs (Build 1 and Build 2) of the same suite.

COMPARISON REPORT:
{comparison}

Definitions you MUST follow:
- FLAKY = non-deterministic result: passed in one build and failed in the other, OR passed only after a retry. Flaky tests need a rerun / quarantine, not a code fix.
- CONSISTENT FAILURE = failed in BOTH builds. A real, reproducible bug, NOT flaky. Needs a fix.

Produce:
1. FLAKY_TESTS - names + one-line hypothesis of flake cause (timing, data, parallelism, network...).
2. CONSISTENT_FAILURES - tests failing in both builds, each with a probable root cause.
3. RERUN_RECOMMENDATION - which to rerun (flaky) vs send to engineering (bugs).
4. SUMMARY - counts + one sentence on suite health.

Base everything only on the comparison data. Do not invent test names.
Note the guardrail. The prompt restates the flaky-vs-bug definitions even though the diff already separated them. This stops the model from "helpfully" re-labelling a consistent failure as flaky in its narrative — a common drift when the LLM thinks it knows better.

One secret to fill (Groq), plus the custom component code is already embedded in the flow.

Generate two Playwright reports

Add the JSON reporter: reporter: [['json', { outputFile: 'results.json' }]] in playwright.config.ts. Run your suite twice (e.g. across two builds) to get build1.json and build2.json.

Import the flow

Download the JSON below → Langflow → Import. The Flaky Diff custom component ships inside it, code included.

Upload both reports

In the two File nodes, upload build1.json to Build 1 and build2.json to Build 2.

Add your Groq key

Paste your gsk_… key into the Groq node. Model preset to openai/gpt-oss-120b.

Run it

Open Playground and run. The report lists flaky tests with cause hypotheses, consistent failures with root causes, and a clear rerun recommendation.

Going hands-off. Replace the File nodes with a Webhook that receives reports from CI, and Chat Output with nodes that auto-open Jira tickets for consistent failures and trigger reruns for flaky ones. Trigger via the flow's /run endpoint at the end of every pipeline.
Download flaky_analyzer.json

Test the analyzer against the failure modes that would mislabel a test.

T01

Mixed = flaky: a test that passes in build 1, fails in build 2. Verify it lands in flaky, never consistent_failures.

T02

Both = bug: a test failing in both builds. Verify it's a consistent failure and the rerun advice sends it to engineering, not a rerun.

T03

Retry pass: a test that passed only after a retry in one build. Verify it's flagged flaky even though both builds are green.

T04

New / removed tests: a test present in only one build. Verify it doesn't crash the diff or get miscounted as failed.

T05

Count integrity: flaky_count + consistent_failure_count must match the arrays. The model must not invent or drop tests.

T06

Malformed JSON: feed a truncated report. Verify the File/diff step fails loudly rather than silently triaging nothing.

2
The output contract

What a run returns

{
  "flaky_tests": [
    "profile loads — pass→fail across builds, likely a race on async profile fetch",
    "search returns results — passed on retry, probable missing wait for results render"
  ],
  "consistent_failures": [
    "checkout completes — failed in both builds, payment callback returns null order ID"
  ],
  "rerun_recommendation": "Rerun profile + search; file checkout as a bug — do not rerun.",
  "flaky_count": 2,
  "consistent_failure_count": 1,
  "summary": "1 real bug in checkout; 2 flaky tests inflating the red. Suite health is recoverable."
}
Agent 03 · Contracts

API Contract Validator

Paste your cURL requests. It authenticates, threads the token through every protected call, fires them for real, and validates each response against a JSON Schema — then reports the verdict.

9
Nodes
Real
HTTP calls
Cookie
Token auth
Schema
Validated
1
Contracts · the problem

Does the API keep its promises?

Objective

An API has a contract: each endpoint should return a known status code and a response that matches an agreed shape — the right fields, the right types. When a deploy quietly changes a field from integer to string, or drops a required key, downstream consumers break. We want to catch that automatically.

The user pastes the cURL requests for an end-to-end flow — like Restful Booker's auth → create → read → update → delete. The agent runs the whole chain for real, carrying the auth token through the protected calls, and validates every response against a JSON Schema. Out comes a clear pass/fail contract report.

The integrity rule, again. The HTTP calls and the schema checks happen in real code — Python requests and the jsonschema library inside a custom component. The LLM never fabricates a 200 OK or a passing validation. It only parses your messy cURLs into a clean plan and explains failures in plain English.
Teach
The closing lesson of the series ties the thread together: an AI agent is most trustworthy when the verifiable work is done deterministically in code, and the LLM is used only where judgement and language actually help. Contract testing makes that split impossible to fake — a hallucinated pass is worse than useless.
Langflow canvas

Two LLM stages bracket one deterministic runner: parse the cURLs → run + validate for real → explain the result.

IN
Chat Input
cURL commands

The user pastes the requests to validate, one per line — auth call first, then the chain.

out: Message
P
Prompt Template
cURL parser brief

Asks the model to turn each cURL into a structured request with its contract expectations (status + schema kind).

in: curlsout: Message
AI
Groq Model
Parse cURLs — gpt-oss-120b

Tolerates messy, real-world cURL syntax and produces a clean structured plan. A good LLM job.

in: promptout: Message + LanguageModel
[ ]
Structured Output
Request plan as JSON

An ordered list of {method, url, headers, body, expects}.

in: text + llmout: Data
Custom Component
API Contract Validator — the real work

Authenticates, extracts the token, injects Cookie: token= on PUT/PATCH/DELETE, fires every request with requests, validates each response with jsonschema. Deterministic — no LLM.

in: request_planout: Data
P
Prompt Template
Report writer brief

Asks the model to explain the real results readably — but it cannot overturn a verdict.

in: reportout: Message
AI
Groq Model
Report Writer — gpt-oss-120b

Turns pass/fail data into a plain-English contract report with next steps.

in: promptout: Message + LanguageModel
[ ]
Structured Output
Contract report JSON

overall_passed, counts, per_call, next_steps.

in: text + llmout: Data
OUT
Chat Output
Show the report

In CI, hit the flow's /run endpoint and fail the pipeline if overall_passed is false.

in: Data
Two LLM stages, one runner. The model bookends the flow — parsing input at the front, explaining output at the back — while the custom component in the middle does everything that must be true. That sandwich is the pattern for any agent that has to act on the world, not just reason about text.

The deterministic core, inside the custom component. Real requests traffic, real cookie threading, real jsonschema validation.

api_contract_validator.pypython
def run_validation(self) -> Data:
    session = requests.Session()
    results, token = [], None

    # 1) AUTH FIRST - get the token. Restful Booker returns 200 even on
    #    bad credentials, so assert the token EXISTS, not just the status.
    auth = session.post(f"{self.base_url}/auth",
                        json={"username": self.username, "password": self.password},
                        timeout=self.timeout)
    token = auth.json().get("token")
    results.append({"name": "AUTH POST /auth", "actual_status": auth.status_code,
                    "passed": auth.status_code == 200 and token is not None})

    # 2) Run the rest of the plan IN ORDER, threading the token through.
    for req in self._plan():
        method, url = req["method"].upper(), req["url"]
        headers = dict(req.get("headers") or {})
        headers.setdefault("Accept", "application/json")
        # cookie auth ONLY on the mutating calls - exactly Restful Booker's contract
        if method in ("PUT", "PATCH", "DELETE") and token:
            headers["Cookie"] = f"token={token}"

        resp = session.request(method, url, headers=headers,
                               json=req.get("body"), timeout=self.timeout)
        expects = req.get("expects", {})
        status_ok = resp.status_code == expects.get("status", 200)
        # REAL schema validation - jsonschema, not the LLM guessing
        schema_ok, errors = self._validate(resp.json(), expects.get("schema_kind", "none"))
        results.append({"name": req["name"], "expected_status": expects.get("status"),
                        "actual_status": resp.status_code, "status_ok": status_ok,
                        "schema_ok": schema_ok, "errors": errors,
                        "passed": status_ok and schema_ok})
    return Data(data={"results": results})
Why auth checks the body, not the status. Restful Booker returns 200 OK even for bad credentials (with {"reason":"Bad credentials"} instead of a token). So the runner asserts the token actually exists before proceeding — a real-world reminder that a green status code is not the same as success.
Teach
Run the component in isolation first with a hardcoded plan. Show a deliberately wrong schema (declare totalprice as a string) producing schema_ok=false with the exact failing field. That's the moment students trust the agent isn't bluffing.

The Restful Booker contract the agent validates against. Status codes and response shapes the live API actually returns.

CallAuthExpectValidates
POST /auth200 + tokentoken is a string
POST /booking200bookingid + booking object
GET /booking/{id}200full booking shape
PUT /booking/{id}Cookie: token=200updated booking shape
DELETE /booking/{id}Cookie: token=201status only

The booking schema

booking.schema.jsonjson schema
firstname        string   (required)
lastname         string   (required)
totalprice       integer  (required)
depositpaid      boolean  (required)
bookingdates     object   (required)
  checkin        string  YYYY-MM-DD
  checkout       string  YYYY-MM-DD
additionalneeds  string   (optional)
Two Restful Booker gotchas worth teaching. DELETE returns 201 Created (not 200/204) — a genuinely odd choice baked into the sandbox. And the live instance only accepts the Cookie token form; the Basic-auth alternative in the docs often returns 403. Validate against what the API does, not just what it documents.

The cURL-parser brief inside the first Prompt Template. The model's only job here is turning messy cURLs into a clean plan — it never makes a call.

curl_parser_prompt.txtprompt
You convert raw cURL commands into a clean JSON request plan for an API contract test runner.

CURL COMMANDS:
{curls}

For EACH cURL, extract: name, method, url, headers, body, and expects (status = expected HTTP code, schema_kind = auth | booking | create | none).

Rules:
- /auth -> status 200, schema_kind 'auth'.
- POST /booking -> status 200, schema_kind 'create' (response is wrapped: bookingid + booking).
- GET/PUT/PATCH /booking/{id} -> status 200, schema_kind 'booking'.
- DELETE /booking/{id} -> status 201, schema_kind 'none'.
- Do NOT add a Cookie/token header yourself - the runner injects the auth token automatically.
Return ONLY the structured plan. Do not execute anything.
Note the explicit instruction not to add the token. The prompt tells the model not to invent a Cookie header — the runner injects the real token after auth. Keeping that responsibility in code, and telling the model to stay out of it, is what stops the LLM from faking authentication.

One secret to fill (Groq). The custom component code and the schemas ship inside the flow.

Import the flow

Download the JSON below → Langflow → Import. The API Contract Validator custom component, with its Restful Booker schemas, is embedded.

Allow the custom component to run

The validator runs Python (requests + jsonschema). If your Langflow has code execution locked down, approve/trust the component on first import. jsonschema ships with most Langflow installs; if you hit an ImportError, add it to your instance's requirements.txt.

Add your Groq key

Both Groq nodes use the same gsk_… key, preset to openai/gpt-oss-120b.

Paste your cURLs

The Chat Input ships with a full Restful Booker example (auth → create → get → update → delete). Replace it with your own API's cURLs, or run the demo as-is.

Run it

Open Playground and send. The runner authenticates, validates the chain for real, and the report lists each call's expected vs actual status and any schema failures.

Going hands-off in CI. Call the flow's /run endpoint at the end of your deploy pipeline, pass the cURLs as input, and fail the build when overall_passed is false. That turns this into a real contract gate between services.
Download api_contract_validator.json

Test the validator against the failures that matter most — the ones where a naive check would pass something broken.

T01

Status mismatch: point a call at an endpoint that returns 500. Verify the report marks it failed with expected-vs-actual, not a pass.

T02

Type drift: validate a response where totalprice comes back as a string. The schema check must fail and name the field.

T03

Missing field: drop a required key like lastname. Verify schema_ok is false with a clear "required property" error.

T04

Token threading: confirm the Cookie: token= header appears only on PUT/PATCH/DELETE, never on auth, POST, or GET.

T05

Bad-credentials trap: use a wrong password. Verify auth fails on missing token even though Restful Booker returns 200.

T06

DELETE status: verify a successful delete expects 201, not 200 — the report must not flag the correct 201 as a failure.

T07

No hallucinated pass: break the network mid-run. The failed call must surface the real error, never a fabricated success.

2
The output contract

What a run returns

{
  "overall_passed": false,
  "passed_count": 4,
  "total": 5,
  "per_call": [
    "AUTH POST /auth — 200, token received — PASS",
    "POST /booking — expected 200, got 200, schema valid — PASS",
    "GET /booking/42 — expected 200, got 200, schema valid — PASS",
    "PUT /booking/42 — FAIL: totalprice returned as string, expected integer",
    "DELETE /booking/42 — expected 201, got 201 — PASS"
  ],
  "next_steps": "Fix the PUT response type contract for totalprice before release; all other endpoints conform."
}
Reference

The Shared Stack

Every agent in this series is built from the same small set of pieces. Learn these once.

The backbone

Six steps, every time

StepLangflow pieceJob
Get the dataAPI Request / File / WebhookPull the raw engineering data in
Make it readableParse Data / Custom ComponentFlatten or pre-process for the model
Say what to decidePrompt TemplateThe reasoning spec + variables
Let it reasonGroq Model · gpt-oss-120bThe actual LLM thinking
Force the shapeStructured OutputProse → strict typed JSON
Ship itChat Output / API RequestDisplay or write back

Why Groq + gpt-oss-120b

Groq's OpenAI-compatible endpoint runs the open gpt-oss-120b model at high speed and low cost — ideal for classification and triage where you call it often and want fast, stable answers. Temperature stays low (0.1) so verdicts don't drift between runs. Swap to llama-3.3-70b-versatile if you prefer.

The one concept beginners miss

Two outputs from the model node. Structured Output needs both the Message (the text to extract from) and the LanguageModel (a live model to do the extracting). Both wires come from the same Groq node into the same Structured Output node. Get this right and the rest of the flow just works.

Reason in code, judge with the LLM

Where logic is deterministic — counting failures, comparing two runs, exact classification — do it in a custom component, not the prompt. Hand the model only the parts that need judgement. It's cheaper, exact, and testable. Agent 2's Flaky Diff is the template for this. Agent 3 goes further with a two-LLM sandwich: the model parses input at the front and explains results at the back, while a deterministic runner in the middle does everything that must actually be true.