The task was simple: run a million-vector benchmark and see whether a recent change to the auto-tuner made any difference. The change had two halves. A floor, so the sample the tuner trains on never shrinks below a minimum. And a gate, so sampling waits for ingest load to settle before deciding how big that sample should be.
It made no difference, and I could prove that without running the benchmark at all.
The sample size comes from banded fractions -- small collections sample 20%, medium collections 5%. A million vectors sits in the medium band, so it samples 50,000. The floor is derived from the small band, and works out to 40,000. Fifty thousand already exceeds forty thousand, so the floor cannot bind. Not "didn't happen to bind on this run" -- cannot, arithmetically, at this size. It only has an effect between roughly 200,000 and 800,000 vectors, a window a million-vector test flies straight past.
The gate did fire, and fired correctly: sampling waited about 73 seconds, a single tick, until after the load finished. But firing correctly is not the same as mattering. The race it prevents is one the earlier runs weren't losing -- their archived plans show full-corpus sampling too. Four archived run ledgers, before and after the change, all show the same thing: 50,000 vectors, 200 queries, across an identical 217-experiment grid.
That's the whole finding, and it took twenty minutes against records that were already on disk instead of several hours of machine time producing a curve that would have looked identical to the last one and told me nothing about why. The benchmark would have measured the change honestly and I'd have concluded it had no effect at a million vectors, which is true but useless -- I'd have learned the same fact without learning that it's a fact by construction, or where the change DOES apply.
Keeping the ledger from every tuning run is a habit that pays off in exactly this shape. The cheapest experiment is frequently one you already ran. And the actionable half of a negative result is the boundary condition it hands you: if anyone wants to genuinely exercise that change, it needs a collection between 200,000 and 800,000 vectors and a deliberately trickled upload. A million-vector run cannot test either half, no matter how many times it's run.