AI Tester Blueprint · Capstone Project

Build Your Own AI QA Copilot

A production-grade, multi-source RAG system every QA engineer should build to actually learn AI — grounded in your code, test cases, PRDs, and bugs.

localhost:5173 — QA Copilot · Chapter 09 RAG · cited sources

QA QA Copilot Chapter 09 💬 ChatRAG ExplorerStatusKT Doc

…Steps: 1. Open the mobile app and tap Login. 2. Authenticate through the external IdP (e.g., Google). Expected: app receives a valid OAuth token and shows the dashboard.

Give me a source code of Selenium where the login module was mentioned.

route: selenium_code · User requests Selenium source code for a login mo… · search 5.1ms · rerank 1161.1ms

Here is the Selenium Java code that implements the login module: // src/main/java/com/thetestingacademy/pages/POM/vwo/normal_POM/LoginPage.java public class LoginPage { WebDriver driver; public LoginPage(WebDriver driver) { this.driver = driver; } // locators private By username = By.id("login-username"); private By password = By.id("login-password"); private By signButton = By.id("js-login-btn"); // login with invalid creds → returns the error text public String loginToVWOInvalidCreds(String usr, String pwd) { driver.get(PropertiesReader.readKey("url")); driver.findElement(username).sendKeys(usr); driver.findElement(password).sendKeys(pwd); driver.findElement(signButton).click();

AnswerGenerate TC

Ask about Selenium / Playwright code, VWO test cases, PRDs, or JIRA bugs…

Send

Source filter

Auto: router decides.

SeleniumPlaywrightTest CasesPRDs / DocsJIRA Bugs

SOURCES FOR LAST ANSWER

1Seleniumrerank 0.003

src/main/java/com/thetestingacademy/utils/PropertiesReader.java:11–26 · method readKey

public static String readKey(String key) { Properties p; try { String user_dir = System.getProperty("user.dir");

2Seleniumrerank 0.002

src/main/java/com/thetestingacademy/pages/POM/vwo/normal_POM/LoginPage.java:9–54 · class LoginPage

public class LoginPage { WebDriver driver; public LoginPage(WebDriver driver) { this.driver = driver;

Real app, real answer. Ask for Selenium login code → QA Buddy routes to selenium_code, returns LoginPage.java, and shows the reranked source chunks it used.

The Path You'll Walk

From QA Engineer → AI QA Engineer

🧑‍💻

QA Engineer

you, today

›

Coding Fundamentals

Python & JS/TS
Git, project structure
Reading frameworks

›

Embeddings & Vector DB

BGE-M3 dense + sparse
Qdrant collections
Similarity search

⤵

Chunking Strategies

tree-sitter (Java/TS)
row-aware CSV
page-aware PDF

›

Hybrid Retrieval

dense + sparse
RRF fusion
top-k recall

›

Reranking

cross-encoder
BGE-Reranker-v2-m3
precision before LLM

⤵

Intent Routing

query rewrite + history
classifier picks collections
multi-source fan-out

›

RAG Orchestration

retrieve → rerank → generate
grounded, cited answers
the RAG Explorer

›

Serving the App

FastAPI + SSE streaming
React + Vite chat UI
source-cited responses

⤵

🐕 QA Buddy — shipped

chat over code, tests, PRDs, bugs
every answer cites its source
runs on your laptop

›

🚀

AI QA Engineer

RAG, shipped & understood

What QA Buddy Is

One chatbot with a brain over your entire QA world. Ask a question in plain English; get a cited answer pulled from the right source — not a hallucination.

Selenium framework

Java repo · chunked per method/class

Playwright framework

TS repo · per test() block

Test cases

CSV · up to 20,000 rows, row-aware

PRDs & PDFs

page-aware chunks (PyMuPDF)

Jira bugs

exported MDs + Jira MCP

Design Principles

Ground every answer — cite the exact source chunk.
Route by intent before you retrieve.
Hybrid retrieval: dense + sparse, not keyword-only.
Rerank with a cross-encoder before the LLM reads it.
Keep it local-first — your data never leaves your laptop.
One collection per source = clean, debuggable retrieval.

Five Qdrant Collections

selenium_code playwright_code vwo_testcases vwo_docs vwo_bugs

The Query Pipeline — Every Question

Rewrite+ chat history

›

Routerpick collection

›

Hybriddense+sparse · RRF

›

Rerankcross-encoder

›

Groq LLMstream

›

Answer+ citations

The retrieval is the product. The LLM only writes up what good retrieval already found.

What You Learn Building It

Retrieval

hybrid dense+sparse
RRF fusion
cross-encoder rerank

Chunking

tree-sitter (Java/TS)
row-aware CSV
page-aware PDF

Orchestration

query rewrite
intent router
multi-collection

Delivery

FastAPI + SSE
streaming chat
RAG Explorer

The Stack

Qdrantvector DB · 5 collections BGE-M3dense + sparse embed BGE-Reranker-v2-m3cross-encoder Groqstreaming LLM FastAPI + SSEbackend React + Vitechat UI tree-sittercode chunking PyMuPDFPDF parsing

Why It Matters for QA

Real AI skill

A system you built and can defend in any interview.

Your data, useful

Years of tests + bugs become searchable.

Source Files Sneak Peek

ingest/ingest_all.py

for src in SOURCES:
    chunks = chunk(src)
    vecs = bge_m3.encode(
        chunks, dense=True,
        sparse=True)
    qdrant.upsert(
        src.collection, vecs)
# 5 sources → 5 collections

rag/router.py

route = classify(question)
hits = []
for col in route.collections:
    hits += hybrid_search(
        col, q_dense, q_sparse)
ranked = reranker.rank(
    question, rrf(hits))  # top-k

backend/main.py

@app.post("/chat")
async def chat(q):
    ctx = retrieve(q)
    return StreamingResponse(
        groq.stream(q, ctx),
        media_type="text/event-stream")
# cited answer, streamed

Retrieval beats prompting.

Good chunks + routing + rerank decide answer quality.

Ground everything.

Every answer cites the chunk it came from — no guessing.

Build it once, own the skill.

RAG stops being a buzzword when you've shipped one.

QA Buddy · The Testing Academy — the AI Tester Blueprint capstone project