The Study

Every chart on this page renders the committed results of tools/bench.py — reproducible with one command, measured at two scales, with scanned vectors (distance evaluations) as the portable cost metric. Wall-clock is single-thread pure Python: shown, but treat it as relative.

loading committed results…

1 · The index dial: recall vs work

Before sharding at all: what does approximate search cost inside one shard? IVF and HNSW both trade recall for scanned fraction — the curves below are that trade, measured at the larger scale. Flat sits at (100% scanned, recall 1.0) by definition.

2 · The five cuts: what a global query costs

Now shard the store eight ways (corpus placement uses its natural shard count) and run the same semantic queries with no scope hints. Balance is max/mean shard size (1.0 = perfect). Unpruned fan-out always reproduces the exact answer — sharding changes cost, never answers — so the interesting column is scanned fraction when a strategy is allowed to prune.

3 · Scoped queries: where placement starts to matter

The parent study's whole thesis was that agent queries arrive with addresses — a domain, a coordinate window — not just a vector. Give the router those hints and the strategies separate sharply:

4 · The resize experiment

Grow the cluster from 8 shards to 9. Under naive modulo hashing, almost every record's placement changes; under rendezvous (consistent) hashing, only the records whose new shard wins the weight function move — in theory 1/9 ≈ 11%. Measured:

5 · Readings