Everything on this page was measured with VectorDBBench — the open-source benchmark harness maintained by Zilliz — running against the same public API every customer uses. No internal shortcuts, no synthetic data, no hand-picked runs. Raw harness output is downloadable below, including the run our own rate limiter spoiled.
VectorDBBench's Cohere 1M case: one million 768-dimension embeddings, cosine distance, recall@100 scored against the dataset's own ground truth. The index was chosen by our automatic optimizer — nobody hand-tuned anything.
| Dataset | Cohere 1M (768d), VectorDBBench standard case, cosine, k=100 |
| Path measured | Public API (api.vectorpanda.com) — TLS, auth, the works |
| Index | Auto-selected by the optimizer (HNSW family); zero manual tuning |
| Recall@100 | 0.9652 (NDCG 0.9667), scored on the dataset's full ground truth |
| Serial latency p99 | 49.5 ms |
| Max sustained QPS | 1,620 (flat from 100 to 200 concurrent clients) |
| Errors during the sweep | 0 across ~236,000 queries |
| Collection price | $17.14/month (2.86 GB hot × $5.99/GB-month, list price — queries included) |
Raw harness output: this run (JSON) · the rate-limited first attempt (JSON)
30 seconds per step, VectorDBBench's multiprocess runner. Throughput reaches its plateau around 100 concurrent clients and holds flat while latency grows linearly — a saturated system queueing politely, with zero failures.
| Concurrent clients | QPS | p95 | p99 |
|---|---|---|---|
| 1 | 46 | 42 ms | 51 ms |
| 10 | 486 | 24 ms | 53 ms |
| 20 | 872 | 28 ms | 59 ms |
| 50 | 1,462 | 41 ms | 83 ms |
| 100 | 1,619 | 88 ms | 138 ms |
| 150 | 1,620 | 156 ms | 198 ms |
| 200 | 1,604 | 179 ms | 247 ms |
Rows below are from VectorDBBench's published July 2026 results for the same Cohere 1M case; monthly prices are each vendor's list price for the configuration benchmarked. Ours is the list price of the collection itself.
| System | QPS | Recall | p99 | $/month | QPS per $/mo |
|---|---|---|---|---|---|
| ZillizCloud 8cu-perf (best row) | 12,838 | 0.9588 | 2.1 ms | $929 | 13.8 |
| OpenSearch 16c128g | 2,590 | 0.9679 | 8.6 ms | $1,035 | 2.5 |
| Vector Panda | 1,620 | 0.9652 | 49.5 ms | $17.14 | 94.5 |
| ElasticCloud 8c60g | 1,482 | 0.9546 | 12.1 ms | $920 | 1.6 |
| Pinecone p2.x8 | 1,146 | 0.9262 | 13.7 ms | $956 | 1.2 |
| QdrantCloud 16c64g | 1,111 | 0.9550 | 7.0 ms | $832 | 1.3 |
| TurboPuffer (serverless) | 798 | 0.8993 | 56.7 ms | — | — |
| S3Vectors (serverless) | 199 | 0.8717 | 337 ms | — | — |
Comparisons only make sense inside a recall band: our 0.9652 sits between Zilliz's best row (0.9588) and OpenSearch (0.9679). Vendors' numbers come from their published VectorDBBench runs on their hardware; we have not re-run their systems ourselves.
Read it honestly. Zilliz and OpenSearch beat us on absolute throughput — if a single collection needs more than ~2,500 QPS today, or you need single-digit-millisecond p99, those are the right tools and they are priced for it. Where the comparison tilts hard the other way is throughput per dollar: this collection serves 1,620 QPS for $17.14 a month — about 94 QPS per dollar-month against 13.8 for the best published row above. Queries are included in the price, so the ratio doesn't decay with traffic.
The harness is open source, the dataset is standard, and the API is the public one. A skeptical afternoon is all it takes.
pip install vectordb-bench) — it downloads the Cohere 1M dataset
and ground truth itself.veep SDK's create/upsert/query calls, nothing more.--prod, load the dataset, let the
optimizer converge, and run the sweep. The collection you create bills like any
other (about $17/month hot for this dataset — delete it when you're done).