Embed & index chunks·20 min
Embed & index
0 / 1
# grade this stage (from your course directory)
npx theinference test --stage embed-and-index --watchBuild the chunk index. Chunk the corpus (recursive chunker), embed every chunk, and assemble the {ids, vectors} index. Then prove it works by retrieving the top-5 chunks for the first question.
What to build
Your task
Build the chunk index
The build-index command chunks the corpus, calls embedText on each chunk, stacks the vectors into the index, and prints the chunk count, the embedding dim (256), and the top5 chunk ids for question 0 retrieved over the index. The wiring is in cli.ts and src/data.ts; implement embedText and cosine/retrieve so it runs.
cli.ts
case "build-index": {
const { chunks, index, qa } = buildPipeline("recursive"); // chunk + embedText each
const top5 = retrieve(embedText(qa[0]!.question), index, 5).map((s) => s.chunkId);
out({ chunkCount: chunks.length, dim: index.vectors[0]!.length, top5 });
break;
}
How the grader runs it
./your_program.sh build-index recursive
# -> { chunkCount, dim: 256, top5: [chunkId, ...] }
How it's graded
process mode, stdout_json_match. The chunk count, dimension, and the top-5 chunk ids must match the reference exactly. Because the embedder is deterministic, your top-5 is the exact reference top-5 — no tolerance needed.