The contrastive loss·6 min
In-batch negatives
0 / 1
Here's the central trick of modern embedding training. To teach the model that query q matches its positive p, you'd ideally softmax over every document in the corpus — impossibly expensive. Instead: take a batch of B (query, positive) pairs, and treat the other B−1 positives as free negatives. Query i must pick its own positive i out of all B positives in the batch.
The B×B similarity matrix
diagonal = the gold label
pos0 pos1 pos2
q0 [ .91 .12 .08 ] ← row 0 should peak on column 0
q1 [ .05 .88 .20 ] ← row 1 should peak on column 1
q2 [ .10 .15 .93 ] ← row 2 should peak on column 2
cross-entropy on each row, target = its own diagonal entry
Why this is genius
You get B−1 negatives per example for free, with no negative-sampling step to implement or reproduce. Determinism reduces to a fixed batch order — there's no random sampler. This is exactly what SimCSE and sentence-transformers do in practice.