You've trained an embedding model — Train your own embedding model

You did it. Random vectors → encoder forward → InfoNCE → hand-derived backprop → Adam loop → beat the baseline on recall@5. You now understand the exact loss behind SimCSE and sentence-transformers, you derived its gradient by hand and validated it with finite differences, and — most importantly — you can measure a representation model honestly instead of eyeballing nearest neighbors.

the whole arc

cosine + L2-norm  →  tokenize + pool  →  MLP head  →  encode
   →  B×B cosine matrix  →  symmetric InfoNCE
   →  backprop by hand (finite-diff validated)  →  Adam loop
   →  recall@5 ≫ baseline  →  embed / nearest CLI

Where to go next: scale d and the vocab, swap the mean-pool for a tiny transformer encoder, add hard negatives (the SimCSE supervised objective with NLI contradictions), or wire this encoder into a retrieval / RAG pipeline. The recipe is the same — only the scale changes.

References