Is it any good? (qrels, MRR, nDCG)·10 min

DCG

0 / 1

# grade this stage (from your course directory)
npx theinference test --stage dcg --watch

MRR only looks at the first hit. nDCG grades the whole ranked list and rewards putting relevant docs higher. Its building block is Discounted Cumulative Gain — sum each result's relevance, discounted by how far down it sits, so rank 1 counts full and rank 10 counts less.

DCG@k (the trec_eval / BEIR convention)

DCG@k = Σ(i=1..k)  relᵢ / log₂(i + 1)

The discount log₂(i+1) means rank 1 divides by log₂2 = 1 (full credit), rank 2 by log₂3 ≈ 1.585, rank 3 by 2, and so on. In code, with a 0-based index i, rank = i+1, so the discount is log₂(i+2).

What to build

Your task

Implement dcgAtK

Implement dcgAtK(gains, k) in src/metrics.ts — gains is the relevance of each ranked result in order; return Σ_{i=0..k-1} gains[i] / log₂(i + 2).

src/metrics.ts

export function dcgAtK(gains: number[], k: number): number {
  // Σ_{i=0..k-1} gains[i] / Math.log2(i + 2)
}

How it's graded

adapter — allclose vs reference on a hand-built gains list, atol=1e-6. With 0-based i, the discount is log₂(i+2), which equals the formula's log₂(rank+1) since rank = i+1.

log₂, not ln

The discount is base-2: Math.log2(x) (or Math.log(x)/Math.LN2). Using natural log scales every DCG by a constant — which cancels in nDCG, so a buggy ln version would pass the nDCG step but fail this raw-DCG step. Use log2 and match both.

MRR: how high is the first hit?IDCG