Recursive & sentence-window chunking·25 min

Sentence-window chunking

0 / 1

# grade this stage (from your course directory)
npx theinference test --stage sentence-window --watch

The third splitter: sentence-window chunking (LlamaIndex SentenceWindowNodeParser). Emit one chunk per sentence, but the chunk's text is that sentence plus its neighbors on each side. You embed a focused center sentence (sharp retrieval) but hand the model the surrounding window (enough context to answer).

What to build

Your task

Implement chunkSentenceWindow

Split the paragraph into sentences with splitSentences. For each sentence i, emit a chunk whose text is sentences [i-window, i+window] joined by a single space, with centerSentence = sentence i. The token offsets span from the first to the last sentence in the window.

src/chunk.ts

export function chunkSentenceWindow(
  paragraphId: string, text: string, window = 3,
): WindowChunk[] {
  // sentences = splitSentences(text)
  // for each i: lo = max(0, i-window), hi = min(n-1, i+window)
  //   text = sentences.slice(lo, hi+1).join(" "),  centerSentence = sentences[i]
  //   startTok = offset of sentence lo,  endTok = offset end of sentence hi
}

How the grader runs it

./your_program.sh chunk sentence-window "dna#p1" 1
# -> one chunk per sentence; window text + offsets compared exactly

How it's graded

process mode, stdout_json_match. Chunk count must equal the sentence count; each window text and the offsets must match the reference exactly.

Sentence segmentation Which chunker when