A vector query is, underneath, just a comparison of one vector against a lot of other vectors. Maybe a few thousand. Maybe a few hundred million. The math itself is fast; the slow part is getting the data the math operates on into the CPU.
That's a memory-hierarchy problem, and it's the same problem CPUs have been solving with L1, L2, and L3 caches for decades: keep the data you'll touch most often closest to the chip, and accept that the data you touch rarely has to live somewhere further away. Vector Panda exposes this trade-off as three tiers a collection can sit in: hot, warm, and paused. The tier you pick determines where your vectors physically live when a query arrives, which determines how fast that query is, which determines what you pay.
Hot: RAM-resident
A hot collection's vectors are loaded into RAM on the worker that owns them. When a query lands, the comparison loop runs over data that's already in memory; the only physical work is the floating-point math. Tail latency on a hot collection is in the single-digit milliseconds for collections up to roughly the size of a worker's RAM.
Hot is the tier you want for any query path that a customer or an application is actively waiting on. Think: a search box on your website, a recommendation widget that loads with the page, a RAG retriever called at every chat turn. If a human or a downstream service is blocked on the result, hot is the right answer.
The cost is RAM. RAM is the most expensive layer of any storage hierarchy, and a worker can only hold so many gigabytes of vectors before it runs out. Hot is priced accordingly.
Warm: SSD-resident, paged in on demand
A warm collection's vectors live on a local SSD. Queries are served by memory-mapping the vector files and letting the operating system's page cache do its job: pages that the query touches get pulled into RAM, pages that don't stay on disk. The first query on a freshly warmed collection might pay an SSD-read penalty (typically a few hundred milliseconds for a sweep of a 10M-row collection); subsequent queries against the same data hit a warm page cache and approach hot-tier latency.
Warm is the right tier when query frequency is meaningful but not constant. Think: an analyst running a few searches a day across a backup catalog, an internal tool that runs a query when someone clicks a button, an enrichment job that runs a few times an hour. The first query of the day might be a hair slower, and that's fine.
The cost is SSD instead of RAM. SSD is roughly an order of magnitude cheaper per gigabyte than RAM, so the same collection costs about a tenth as much warm as it does hot.
Paused: cold storage
A paused collection's vectors are archived to a slower, denser storage layer (in our case a local ZFS pool with compression). The data is still safe and intact, but it isn't loaded anywhere a worker can query immediately. To query a paused collection, the system has to rehydrate it back into a warm or hot tier first, which can take seconds or minutes depending on size.
Paused is the right tier for data you want to keep around but don't expect to query often. Think: last quarter's product catalog you might pull up if a customer asks about a discontinued item, a snapshot of your support ticket corpus from before a schema change, an embedding archive you keep for compliance. Anything where the answer to "when's the next time I'll query this" is "I don't know, but probably not this week."
The cost is the lowest of the three. ZFS-compressed cold storage is cheap enough that pausing a collection you're not actively using is almost always the right move financially.
Switching tiers
A collection's tier isn't a permanent decision. You can promote (paused → warm → hot) or demote (the reverse) at any time. Promotion takes a moment of work to load the data into the new tier; demotion is essentially free. Billing changes immediately in both directions: the moment you upgrade, you pay the new rate; the moment you downgrade, you stop paying the old one.
This makes the cost dial usable. If you have a dataset that gets heavy traffic during business hours and almost none at night, you can keep it hot during the day and demote to warm overnight. If you launch a feature that turns out to need more responsiveness than you'd budgeted for, promote with one call and the next query is already faster.
How to pick
The instinct is to pick by data size: small things go hot, big things go warm. Resist that. Pick by access pattern instead.
A 10-million-row collection that gets queried once a week belongs in warm, regardless of how big it is. A 10,000-row collection that backs a search bar belongs in hot, regardless of how small it is. The cost of hosting hot data scales with size, but the cost of failing to host hot data (a slow user-facing query path) scales with how often it ruins someone's experience.
Most workloads start hot. If the bill grows uncomfortable and you can identify queries that don't need millisecond latency, demote. If you have data that's truly dormant, pause it. The three tiers exist so you can make those decisions individually per collection rather than as one global trade-off.