Bake-off, CLI & scorecard·3 min

You've built RAG

0 / 1

You built the retrieval + assembly half of a real RAG system, end to end: chunk (three ways) → embed → retrieve (recall@k / MRR) → MMR dedup → answer-span oracle → assemble + ground + cite → cassette generate → citation integrity → a measured bake-off → a scorecard. You can now debug a real RAG system by number, not vibe.

The two failure modes you can now separate

Most teams can't tell these apart:

"Answer not retrievable" — the gold span never made the top-k. You measure it with hit-rate@k and recall@k, deterministically, with no generation.
"Citation unfaithful" — the answer is right but the cited chunk is wrong. You catch it with citationPrecision, which an LLM-judge eval would miss.

Separating them is the difference between guessing and fixing.

References

Lewis et al. 2020 — Retrieval-Augmented Generation
Rajpurkar et al. 2016 — SQuAD
LangChain — RecursiveCharacterTextSplitter
LlamaIndex — node parsers (SentenceWindow)
Carbonell & Goldstein 1998 — MMR
sentence-transformers/all-MiniLM-L6-v2

RAG isn't magic. It's chunk, embed, retrieve, assemble — and now you can prove yours works.

Pipeline scorecard