Build your own vector database

What you'll build

You'll build a working vector database in TypeScript by implementing HNSW directly from Malkov & Yashunin's 2016 paper — all five algorithms, from the seeded level dice and priority queues up to the layered insert and tower search. Index 100k real SIFT vectors and hit 95%+ recall@10 while touching under 1% of the database per query, verified against brute-force ground truth under a hard distance-comparison budget. No library — just the graph, the heaps, and the math, so the index you run is the one you wrote.

• A from-scratch HNSW index — heaps, seeded level assignment, SEARCH-LAYER, both neighbor-selection strategies, INSERT, and K-NN-SEARCH — that searches 100k SIFT vectors at 95%+ recall@10 under a frozen comparison budget
• Payload filtering that stays correct where the naive approaches break: it traverses the full graph for connectivity but only admits matching nodes, with a planner fallback to full scan for selective filters
• A binary segment format that serializes the graph, vectors, and payload to disk and reloads them byte-for-byte, with identical search results before and after
• Int8 scalar quantization with rescoring — 4x smaller vectors searched in integer space, recall held within budget — wired into a mini-Qdrant CLI that builds, filters, persists, and reloads

What you'll learn

• How an approximate-nearest-neighbor index actually navigates — a multi-layer proximity graph behaving like a skip list in metric space, not a black-box index.query() call
• Why neighbor diversity beats keeping the closest M, and how the selection heuristic stops a graph from fragmenting into unreachable pockets on clustered data
• The recall / latency / memory trade-off every vector-DB knob exposes — why efSearch trades recall for comparisons, why a restrictive filter forces a full scan, and what an int8 rescore recomputes
• How to make an 'approximate' algorithm deterministic and verifiable — fixing the seed and insertion order so a correct build is a pure function graded against frozen ground truth

Prerequisites

• Comfortable reading and writing TypeScript, including typed arrays like Float32Array
• Basic linear algebra and big-O intuition — vectors, Euclidean distance, and why O(N) scanning doesn't scale
• Helpful but not required: prior exposure to embeddings or a vector search index (the semantic-search course is the natural lead-in)