Build your own RAG pipeline
Load the corpus·8 min

Load the corpus

0 / 1
# grade this stage (from your course directory)
npx theinference test --stage load-the-corpus --watch

Your first build, and your first green check on real data. Read the corpus, count it, and print a fingerprint of paragraph 0 — proving you parsed the file and the token counter works.

What to build

Your task

Implement the load-corpus command

The load-corpus command reads the corpus JSON and prints one JSON object: the paragraph count, the totalTokens (the sum of countTokens(text) over every paragraph), and p0 — a fingerprint of paragraph 0 as "<paragraphId> | <first 40 chars of text>".

This is already wired in cli.ts; it calls loadCorpus and countTokens (both provided). Just make sure your src modules are in place and run the grader.

cli.ts
case "load-corpus": {
  const corpus = loadCorpus(argv[1]);          // path passed by the grader
  const totalTokens = corpus.reduce((a, p) => a + countTokens(p.text), 0);
  const p0 = corpus[0]!;
  out({ count: corpus.length, totalTokens,
        p0: `${p0.paragraphId} | ${p0.text.slice(0, 40)}` });
  break;
}
How the grader runs it
./your_program.sh load-corpus {fixture}/fixtures/data/corpus.json
How it's graded

process mode, stdout_json_match against expected.json. Your printed count, totalTokens, and p0 string must match the reference exactly — integer-exact counts, exact preview string.

Preserve file order

Ids are assigned in file order and every later stage keys on them. Don't sort or de-duplicate the paragraphs — read them as-is.