The training loop·4 min
The batcher
0 / 1
The trainer reads (anchor, positive) pairs, tokenizes both sides, pads each side to the batch max, and yields fixed-order batches. Order is a seeded permutation recorded in the config — that single seed is what makes the whole training run reproducible to the last decimal.
one batch → loss
batch = { qIds:[B×Tq], qMask, pIds:[B×Tp], pMask }
→ encode both sides → Q [B×32], P [B×32]
→ S = Q·Pᵀ [B×B] → InfoNCE → loss
Batch size sets the difficulty
With batch size B, every query competes against B−1 in-batch negatives, and the loss starts near ln(B) (uniform guess over B classes). Bigger batches = harder, more informative negatives — one reason real recipes push B into the hundreds. We use a small B so it grades fast.