Train your own embedding model
The contrastive loss·35 min

InfoNCE loss

0 / 1
# grade this stage (from your course directory)
npx theinference test --stage infonce-loss --watch

Now the loss itself. Scale S by 1/τ, row-softmax, and take the negative log of the diagonal probability — that's the one-directional InfoNCE that the default MultipleNegativesRankingLoss and SimCSE Eq. 4 use. We then do it both directions (query→positive and positive→query) and average. That symmetric form is sentence-transformers' separate MultipleNegativesSymmetricRankingLoss — a deliberate upgrade over the one-directional default, which we grade against.

symmetric InfoNCE (NT-Xent form)
scores = S / τ                                  # [B,B]
Lq = mean_i ( -log softmax(scores[i])[i] )      # row direction
Lp = mean_j ( -log softmax(scoresᵀ[j])[j] )     # column direction
loss = (Lq + Lp) / 2

What to build

Your task

Implement InfoNCE

Implement runInfoNCE(S, tau) returning [loss] (a one-element number[]). Scale S by 1/τ, row-softmax with the per-row max subtracted (numerical stability), take cross-entropy vs the diagonal, then average the row and column directions. Internally compute the row-softmax matrix too — you'll reuse it for the gradient in chapter 4.

tests/adapters.ts
export function runInfoNCE(S: number[][], tau: number): number[] {
  // scale by 1/tau, row-softmax (subtract row max), CE vs diagonal,
  // average row + column directions; return [loss]
}
How it's graded

Adapter mode: the scalar loss (wrapped as [loss]) is compared to the reference with allclose, atol 1e-6, rtol 1e-5. The fixture ships a fixed S and τ = 0.05.

Numerical stability

Subtract the per-row max before exp, or scaled logits like 20 will overflow Math.exp. This is the same stable-softmax trick used in the attention block of a transformer — never exp a raw logit of 18 without subtracting the max first.

TemperatureThe clean gradient