Embedding dimensions, advanced RAG with a re-ranker and HyDE, modular and graph RAG, and QABuddy, a hybrid RAG for QA

• Why last week's Langflow RAG failed: the embedding and collection dimensions
• The dimension rule of thumb, with the common embedding models
• A recap of naive RAG: ingestion and retrieval
• Advanced RAG: a re-ranker after retrieval
• Advanced RAG: HyDE before retrieval
• Modular RAG, with a smart router
• Graph RAG for source code, and agentic RAG
• Self-RAG, corrective RAG, and BM25 as an interview topic
• The final project: QABuddy, a hybrid RAG for QA
• The QABuddy project as pushed to the repo

Last week's Langflow build stopped at the vector store. The class traced it to one setting: the embedding model's dimension did not match the collection's dimension.

An embedding model turns every chunk into a fixed-length list of numbers, its dimension. A collection in a vector database is created with a dimension too, and every vector stored in it must have exactly that length. Use a 3072-dimension model with a collection created at 1024, and nothing lands.

Three more things to check when a Langflow RAG will not run:

• The database token is valid, and the database itself has actually been created.
• Langflow Desktop is slow to list databases and collections: anywhere from a couple of minutes to a quarter of an hour. Waiting is sometimes the fix.
• Ingestion and retrieval use the same embedding model.

The fixed flow reads the file, structures the raw content with a parser, splits it into 1,000-character chunks, and stores them in Astra DB. The fixed flow is in the batch repo as Fixed_AI4X_Naive_RAG.json, beside the 500 test cases. The advanced RAG flow from this class is there too, as Ai4x_Advance_RAG.json: it adds a Cohere re-rank component and a second language model to the retrieval side.

The rule of thumb: look up your embedding model's dimension first, then create the collection with exactly that number.

Any vector database works the same way: Pinecone, Qdrant, ChromaDB, Milvus. Free embedding models include BGE, Nomic and Qwen.

Naive RAG has two halves. Ingestion: read the documents, parse and clean them, split them into chunks, embed each chunk, and store the vectors. Retrieval: embed the question with the same model, fetch the closest chunks, and pass them to the LLM with the question to write the answer.

Advanced RAG keeps ingestion exactly as it is. Both of its additions are on the retrieval side.

A vector database returns the chunks most similar to the question, the top K, say 20. They are similar, but not ranked by how well they answer the question. A re-ranker reads the question and each of those 20 chunks together, scores them, and passes only the best three or four to the LLM as context.

• Paid: Cohere Rerank is the best-known; a free trial key comes from the Cohere dashboard. Pinecone also sells one, and some vector databases have re-ranking built in.
• Free: BGE re-rankers, Qwen re-rankers, and models served through Ollama.

A question like "give login TC" is a poor search query. The vector database has little to match it against. The class's step before retrieval was an LLM that turns the weak question into better search text, such as "test cases for the login feature, including valid and invalid credentials". In the class's words, it is a better prompter than you, so ask it first.

Strictly, HyDE (hypothetical document embeddings) has the LLM write a hypothetical answer to the question, and searches with that answer's embedding, because an answer looks more like the stored chunks than a question does. Rewriting one question into several better ones has its own name, multi-query or query expansion. Both happen before retrieval, for the same reason, and an interviewer may ask you to tell them apart.

Ingestion is unchanged here too. Modular RAG keeps several collections, such as API, UI and performance test cases, and puts an LLM in front of them as a router. The router reads the question and sends it to the right collection. The Langflow demo routed between two collections, positive and negative test cases, using a smart router component. You choose the model that does the routing, such as GPT, Gemma or DeepSeek.

• Graph RAG is for source code. Code does not chunk well as plain text, so it is parsed into an abstract syntax tree (AST) and stored in a graph database such as Neo4j, where its structure can be searched.
• Agentic RAG puts an agent in place of the router. The agent decides when to retrieve, which collection to search, and when it has enough information, following the ReAct pattern of reasoning, then acting. It returns later in the course, when RAG systems are tested with DeepEval.

The class described self-RAG and corrective RAG as rarely used in practice. The type you will meet most is hybrid RAG, and the final project is one.

The plan from class: a QABuddy, or QA copilot, that ingests Jira, PDFs, the Selenium and Playwright repositories and meeting transcripts, and answers questions about all of them in one chat. It combines normal RAG for the documents with graph RAG for the code, which is what the class meant by a hybrid RAG. Keeping it up to date, and stopping old and new versions of a ticket from conflicting, is handled with metadata.

The class built it with Claude. Command Code, OpenCode, or Antigravity with its free Gemini quota work as well.

The build started from a brief written in class, pushed as QABUDDY_AI_BUILD_.prompt.md. In it, the instructor asks the agent to recommend an open-source embedding model, an open-source vector database, the chunk sizes and overlaps, and a plan for a chatbot running on a DigitalOcean machine. The brief also lists the ten data sources and puts hourly auto-ingestion in phase two. A cleaned-up version, QABUDDY_Improved_.prompt.md, sits beside it.

The result is in the batch repo as chapter_12_RAG_QA_BuddyAI: a Python pipeline, a React chat UI, deployment files and an evaluation set. It is self-hosted: the embedding model and the vector database run on your own machine or server.

The repo's README records the five decisions the brief asked for:

A few chunking choices worth copying, from the README's per-source table:

Two differences from the plan. The graph RAG shows up as code-aware chunking: the code is parsed by structure with tree-sitter and stored in the same Qdrant collection, not in a separate graph database. And "hybrid" gains a second meaning, hybrid search: every question runs dense and BM25 search together and fuses the two rankings.

The repo includes a 24-question evaluation set. The README's table comes from the recorded run, eval/last_run.json, and the two match:

Hit@6 asks whether the right chunk appears anywhere in the top six. MRR (mean reciprocal rank) rewards putting it near the top: 1.0 means always first. Every retriever found the answer, so the difference is in the ranking. BM25 beats dense search here, which is the point of the earlier note. Take the README's example "Is testLoginPositiveVWO flaky?": by meaning, the ticket that answers it ranks 31st; by keyword, 2nd.

The repo also shows how that answer was assembled. The retrieval trace lists each candidate chunk with its rank from dense search, from BM25 and after fusion, then the re-ranker's score and the final decision:

Try it: the hosted demo is at qabuddy-ai-nine.vercel.app. A public page cannot reach a self-hosted Qdrant, Ollama or re-ranker, so the demo replays answers recorded through the full local pipeline, retrieval traces included. New questions are answered with BM25 in the browser plus a serverless function, limited to 20 questions an hour per visitor.

Run it: ./run.sh starts Qdrant and Ollama, ingests on first run, and opens the UI locally. ./run.sh test runs the 18 regression tests, each pinning a bug found while building; they need no database or model and pass as pushed. ./run.sh eval re-runs the retrieval evaluation.

The README is frank about its limits. Groq's free tier serves about two answers a minute. The 4B embedding model is slow on a CPU-only server. The Docker deployment has not been run end to end. And grounding rules reduce model errors without removing them, which is why every answer cites its sources.

• QABuddy: the code, sample data and prompts are in the batch repo. Clone it with --recurse-submodules to fetch the two framework repositories it indexes.
• The advanced MCP session is being organised, probably this week.