Documentation

New to vector search? Start with the guides and demos just below — plain-language concepts, focused tutorials, and end-to-end builds. Looking for a specific call? The full Python SDK and HTTP API reference are further down, one runnable example per method.

Concepts

Short, plain-language reads on the ideas behind vector search. No SDK code. New to all of this? Start with the first one.

Concept

When to use vector search — and when not to

The starting point: which problems vectors are the right tool for, which ones they're the wrong tool for, and how to tell the difference. No code, no math.

~9 min read
Concept

What's actually inside a vector

Build one by hand from a spreadsheet of animals and the shape stops being mysterious — similarity falls out on its own, and you see why it's a list of numbers.

~8 min read
Concept

A search is just another row

The spreadsheet's payoff: turn your question into a row, drop it in, and read off the nearest rows. A vector search, done by hand.

~8 min read
Concept

How a computer learns what words mean

No dictionary, no rules — just a machine that read the internet and noticed which words keep the same company. The black magic behind embeddings, explained.

~8 min read
Concept

Why you can't read a vector

A real embedding looks like noise — no "meaning" column anywhere. Why the numbers are unreadable, and where the meaning actually hides: in directions, not columns.

~9 min read
Concept

Vector embeddings 101

What an embedding is, why it captures meaning, and how to choose a model for your data.

~10 min read
Concept

Embeddings beyond text: images, audio & video

The same coordinates trick works on photos, sound, and video — and when they share one map, you can search images by typing a sentence.

~11 min read
Concept

Cosine similarity vs Euclidean distance

Why cosine is the default for semantic search, and what changes if you switch metrics.

~6 min read
Concept

Why vector search is fast — and a bit approximate

How you find the closest match among a billion items in milliseconds: you don't check all billion. The shortcut, and the honest trade it makes.

~7 min read
Concept

How AI assistants answer questions about your docs

The AI was never trained on your files, so how does it answer accurately without making things up? The open-book trick, and where vectors fit.

~8 min read
Concept

When good embeddings give weird results

Your search returns nonsense and nothing looks broken. Usually it's one of four common, fixable mismatches. How to spot each one.

~7 min read
Concept

How auto-optimize picks your index

Eleven indexing strategies, recall vs latency on a sample of your real vectors, and the Pareto frontier that decides which one wins.

~10 min read
Concept

Hot, warm, paused: a tier mental model

RAM vs SSD vs ZFS as physical layers, and how the tier you pick maps to query latency and cost.

~8 min read

Projects

End-to-end builds (30–60 minutes). Pick a use case, follow it through schema design, ingest, and query.

Tutorials

Focused walkthroughs (5–15 minutes) covering one SDK feature at a time.

Installation

The SDK is published as veep on Test PyPI during the alpha. Python 3.9+.

# Install from Test PyPI with everything the quickstart needs
# (samples = bundled embedding model; pandas = DataFrame upload support)
pip install --index-url https://test.pypi.org/simple/ \
            --extra-index-url https://pypi.org/simple/ "veep[samples,pandas]"

Quickstart

A complete round-trip: log in, create a collection, upload the bundled corpus, run a query.

from veep import VP, samples

vp = VP.login()                                          # browser device flow, saves credentials

vp.collections.create("movies", tier="hot")
vp.vectors.upsert("movies", dataframe=samples.dataframe())   # 5,000 movies, embedded

q = samples.encode("a hobbit destroys a magic ring")
results = vp.vectors.query("movies", vector=q, top_k=3)
for r in results:
    print(f"{r.score:.3f}  {r.metadata.get('title','?')}")
Bundled corpus

samples.dataframe() returns 5,000 Wikipedia movie plots already embedded with the all-MiniLM-L6-v2 ONNX model. samples.encode(text) uses the same model so query vectors land in the same space. Useful for verifying installation; for production, embed your own data with whichever model you prefer.

Python SDK reference

Every public method, in the order a typical script calls them: authenticate, create a collection, upload, query.

Authentication

Four ways to get an authenticated VP instance, ranging from interactive (good for notebooks) to environment-driven (good for CI).

VP.login() — interactive device flow

VP.login(host=None, save=True, open_browser=True, timeout_s=600)

Class method. Opens a browser-based device-code flow, prompts you to confirm, then returns an authenticated client. By default it writes the credentials to ~/.veep/credentials.json so subsequent scripts can use VP.from_creds().

from veep import VP

vp = VP.login()              # prints a code, opens https://www.vectorpanda.com/cli-auth
# vp is now authenticated and ~/.veep/credentials.json is written

VP.from_creds() — reuse saved credentials

VP.from_creds(timeout=120.0, upload_timeout=None, verbose=False)

Class method. Reads ~/.veep/credentials.json (written by a prior VP.login() or vp.save()) and returns an authenticated client. The path used for non-interactive runs.

vp = VP.from_creds()
print(vp.host)            # https://api.vectorpanda.com (or VEEP_HOST override)

VP(api_key=…) — explicit constructor

VP(api_key=None, host=None, timeout=120.0, upload_timeout=None, verbose=False)

Direct constructor. Reads the API key from the VEEP_API_KEY environment variable (or pass api_key= explicitly). The host argument falls back to VEEP_HOST, then to https://api.vectorpanda.com. Raises AuthError when no key is available. This is the recommended shape for CI and other non-interactive environments.

# export VEEP_API_KEY=veep_live_... (set in your shell or CI secrets)
vp = VP()

vp.save() — persist credentials

vp.save()

Writes the current api_key and host to ~/.veep/credentials.json with mode 0600. Useful when you constructed VP programmatically and want subsequent shells to pick it up via VP.from_creds().

# vp constructed any of the ways above
vp.save()

# later, from a different process:
vp2 = VP.from_creds()

Health

vp.ping() — liveness check

vp.ping() -> bool

Hits the cluster's /api/v1/health endpoint and returns True on a 2xx response. The endpoint is unauthenticated by design — it tells you whether the cluster is reachable, not whether your API key is valid. Use vp.whoami() for the auth check.

assert vp.ping() is True

vp.whoami() — identity / auth check

vp.whoami() -> dict

Authenticated equivalent of ping(). Returns the resolved client_id and client_name. Raises AuthError on a missing or invalid API key. Use this when you want to confirm a key is valid before doing real work, without the side effects of collections.list().

info = vp.whoami()
print(info["client_id"], info["client_name"])

Collections

A collection is a named bucket of vectors with a single schema. Tiers (hot / warm / paused) control where the data lives and what queries cost.

collections.create() — create a collection

vp.collections.create(name, *, tier="hot", id_field=None, vector_field=None, format=None, dimension=None, if_exists="ignore")

Creates a new collection. tier defaults to "hot" (RAM-resident). The schema fields are optional — by default the SDK infers them from the first upload.

vp.collections.create("movies", tier="hot")

# with explicit schema hints
vp.collections.create(
    "movies",
    tier="warm",
    id_field="id",
    vector_field="vector",
)

if_exists controls what happens when a collection with this name already exists:

  • "ignore" (default) — log a warning and return the existing collection. Re-running a setup script doesn't fail. The tier and schema args are not applied to the existing collection.
  • "replace" — DESTRUCTIVE. Delete the existing collection (and all its stored vectors) before creating the fresh one. Use only when you intentionally want a clean slate.
  • "error" — raise CollectionAlreadyExistsError. Strict opt-in for callers that need fail-fast on conflict.
from veep.exceptions import CollectionAlreadyExistsError

# idempotent re-run (default) — returns existing if it's there
vp.collections.create("movies")

# wipe + recreate (DELETES ALL STORED VECTORS)
vp.collections.create("movies", if_exists="replace")

# strict mode — raises CollectionAlreadyExistsError on conflict
try:
    vp.collections.create("movies", if_exists="error")
except CollectionAlreadyExistsError:
    pass  # the collection already exists — handle however you like

collections.list() — list every collection

vp.collections.list() -> list[Collection]

Returns every collection visible to the API key. Each Collection exposes name, status, tier, and tier-specific metadata.

for c in vp.collections.list():
    print(c.name, c.status, c.tier)

collections.get() — fetch one

vp.collections.get(name) -> Collection

Returns a single Collection by name, or raises CollectionNotFoundError.

col = vp.collections.get("movies")
print(col.name, col.tier, col.status)

collections.status() — short status string

vp.collections.status(name) -> str

Returns just the lifecycle string: awaiting_artifacts (no data yet), processing, distributing, awaiting_workers, or ready. Cheap polling target after an upload.

import time

while vp.collections.status("movies") != "ready":
    time.sleep(2)

collections.delete() — drop a collection

The example uses a throwaway "scratch" collection so the page's other examples (which all reference "movies") keep working. Substitute the name you actually want to drop.

vp.collections.delete(name) -> None

Drops the collection and every vector in it. Irreversible.

vp.collections.create("scratch", if_exists="replace")
vp.collections.delete("scratch")

Schema

Each collection has one schema: which column holds the ID, which holds the vector, plus dimension and format. Most callers let the schema auto-confirm from the first upload; these methods are for explicit control.

schema.get() — current schema

vp.schema.get(collection) -> SchemaInfo

Returns a SchemaInfo with state, id_field, vector_field, dimension, and format.

schema = vp.schema.get("movies")
print(schema.state, schema.id_field, schema.vector_field, schema.dimension)

schema.confirm() — pin the schema

vp.schema.confirm(collection, *, id_field, vector_field, format=None, dimension=None)

Locks in the field names and (optionally) format/dimension. Only needed if you skipped id_field / vector_field in collections.create() and want to commit them before the first upload.

vp.schema.confirm(
    "movies",
    id_field="id",
    vector_field="vector",
)

schema.update() — change a field name

vp.schema.update(collection, *, id_field=None, vector_field=None, reprocess=False)

Updates one or both schema fields after the schema is already confirmed. Pass reprocess=True to re-extract artifacts under the new schema.

Vectors

The data plane: how rows get in, how queries run, and how individual rows are amended or removed.

vectors.upsert() — four ways to upload

vp.vectors.upsert(collection, file_path=None, *, vectors=None, dataframe=None, table=None)

One method, four input shapes. Pick whichever fits your data layout.

import pyarrow as pa
from veep import samples

df = samples.dataframe().head(100)            # pandas DataFrame, fully embedded
df.to_parquet("plots.parquet", index=False)
arrow_table = pa.Table.from_pandas(df)

# 1. From a pandas DataFrame
vp.vectors.upsert("movies", dataframe=df)

# 2. From a file on disk (.parquet, .csv, .npy, ...)
vp.vectors.upsert("movies", "plots.parquet")

# 3. From a pyarrow Table
vp.vectors.upsert("movies", table=arrow_table)

# 4. Inline rows (small batches, ~ms latency via WAL fast path)
vp.vectors.upsert("movies", vectors=[
    {"id": "m-extra-1", "vector": samples.encode("Aliens"),
     "metadata": {"title": "Aliens", "year": 1986, "genre": "horror"}},
])
Returns an UploadResult

Every mode returns an UploadResult with status (e.g. created / accepted) and filename (the server-side file the upload was stored as). Inline upserts go through a separate WAL fast path; the other three become parquet on the artifact server.

vectors.replace() — overwrite an upload

vp.vectors.replace(collection, file_path) -> UploadResult

Like upsert() with a file path, but instead of adding rows it replaces the contents of the same server-side file. Idempotent for same-content reuploads.

vp.vectors.replace("movies", "plots.parquet")

vectors.query() — similarity search

vp.vectors.query(collection, vector, *, top_k=10, min_score=0.0, with_metadata=True, filter=None, use_index=None, index_params=None, metric="cosine")

Runs a vector similarity search. Results are returned in descending score order. Cosine similarity scores fall in [0, 1]: 1.0 is identical, 0.0 is orthogonal.

q = samples.encode("a hobbit destroys a magic ring")

results = vp.vectors.query(
    "movies",
    vector=q,
    top_k=5,
    min_score=0.4,
    filter={"genre": "fantasy"},
)
for r in results:
    print(f"{r.score:.3f}  {r.metadata.get('title','?')}")
Filter syntax

The filter argument accepts MongoDB-style predicates. Equality ({"genre": "fantasy"}), comparison ($eq, $ne, $gt, $gte, $lt, $lte), set membership ($in, $nin), and boolean composition ($and, $or). Filters apply to metadata fields stored alongside each vector.

metric — only cosine is wired through

The metric parameter accepts "cosine" today. Euclidean and dot-product variants are roadmap; the API will accept the strings but currently routes them through cosine.

vectors.query_batch() — many queries in one round trip

vp.vectors.query_batch(queries) -> list[QueryResults]

Sends a list of query specs in a single request. Each entry is a dict with the same fields as query(): collection, vector, top_k, etc. Returns one QueryResults per input, in order.

prompts = [
    "a hobbit destroys a magic ring",
    "scientists clone dinosaurs and the dinosaurs escape",
    "an off-duty cop is trapped in a high-rise at christmas",
]
batch = vp.vectors.query_batch([
    {"collection": "movies", "vector": samples.encode(p), "top_k": 1}
    for p in prompts
])
for p, hits in zip(prompts, batch):
    print(p, "->", hits[0].metadata)

vectors.fetch() — by ID

vp.vectors.fetch(collection, key, *, with_metadata=True) -> FetchResult

Looks up a single row by its ID. Returns a FetchResult with found, vector, and metadata.

row = vp.vectors.fetch("movies", "m-42")
if row.found:
    print(row.metadata, len(row.vector))

vectors.update() — modify rows in place

vp.vectors.update(collection, vectors) -> dict

Replaces existing rows by ID. Each entry has the same shape as inline upsert(). Returns a dict containing updated_count.

new_vec = samples.encode("Aliens, the director's cut")
vp.vectors.update("movies", [
    {"id": "m-extra-1", "vector": new_vec,
     "metadata": {"title": "Aliens (Director's Cut)", "year": 1986, "genre": "horror"}},
])

vectors.delete() — by file or by IDs

vp.vectors.delete(collection, filename=None, *, ids=None)

Deletes either an entire uploaded file or a specific list of row IDs. Pass exactly one of filename (positional) or ids=.

# by file
vp.vectors.delete("movies", "plots.parquet")

# by IDs
vp.vectors.delete("movies", ids=["m-42", "m-43"])

vectors.list_files() — what's been uploaded

vp.vectors.list_files(collection) -> list[FileInfo]

Returns one FileInfo per uploaded file, with name, size, and metadata. Useful for confirming an upload landed before issuing queries.

for f in vp.vectors.list_files("movies"):
    print(f.name, f.size)

Samples

A bundled toy corpus and a matching encoder. Useful for trying things end-to-end before you wire up your own data.

samples.dataframe() — bundled corpus

samples.dataframe() -> pd.DataFrame

Returns a 5,000-row pandas DataFrame of Wikipedia movie plots, each with a 384-dimensional embedding from all-MiniLM-L6-v2. Columns: id, title, year, genre, plot, vector.

from veep import samples

df = samples.dataframe()
print(len(df), df.columns.tolist())
# 5000 ['id', 'title', 'year', 'genre', 'plot', 'vector']

samples.encode() — encoder for query text

samples.encode(text: str) -> list[float]

Embeds a single string with the same all-MiniLM-L6-v2 ONNX model used by the bundled corpus. Returns a 384-dimensional list of floats. Useful for query vectors when you want them to land in the same vector space as samples.dataframe().

q = samples.encode("a hobbit destroys a magic ring")
assert len(q) == 384

HTTP API

The same surface area as the Python SDK, exposed as REST. Use this for non-Python clients or for environments where adding a dependency isn't possible.

Base URL & authentication

https://api.vectorpanda.com

Every request authenticates with an API key. Most endpoints accept the key via the Authorization header; POST /api/v1/query additionally accepts an api_key field in the JSON body.

Endpoint groupAuth method
All /api/v1/*Authorization: Bearer <api_key>
POST /api/v1/queryHeader above, or api_key in JSON body
GET /api/v1/healthNone — public
GET /api/v1/whoamiHeader above (returns 401 on bad key)

POST/api/v1/query

Run a similarity search. Same semantics as vp.vectors.query().

curl https://api.vectorpanda.com/api/v1/query \
  -H "Content-Type: application/json" \
  -d '{
    "api_key": "veep_live_...",
    "collection": "movies",
    "vector": [0.12, 0.04, ...],
    "top_k": 5,
    "min_score": 0.4,
    "filter": {"genre": "fantasy"}
  }'

Response shape:

{
  "results": [
    {"id": "m-42", "score": 0.84, "metadata": {"title": "Aliens"}},
    ...
  ]
}

GET/api/v1/health

Cluster health. No authentication. Returns 200 OK when the coordinator is up. Use this for status pages and uptime probes.

curl https://api.vectorpanda.com/api/v1/health

GET/api/v1/whoami

Identity / auth check. Authenticated; returns the resolved client_id and client_name in JSON. Returns 401 with the standard error envelope when the key is missing or invalid.

curl https://api.vectorpanda.com/api/v1/whoami \
  -H "Authorization: Bearer veep_live_..."

Collections

MethodPathPurpose
GET/api/v1/collectionsList collections
POST/api/v1/collectionsCreate a collection (JSON body: name, tier, optional schema fields)
GET/api/v1/collections/:nameGet one collection
DELETE/api/v1/collections/:nameDelete a collection

Files (uploads)

MethodPathPurpose
POST/api/v1/collections/:name/files/:filenameUpload a vector file (raw bytes, content-type application/octet-stream)
PUT/api/v1/collections/:name/files/:filenameReplace an existing file
DELETE/api/v1/collections/:name/files/:filenameDelete a file

Schema

MethodPathPurpose
GET/api/v1/collections/:name/schemaGet the current schema
POST/api/v1/collections/:name/schema/confirmPin schema fields

Errors

Errors come back as JSON with an error field. The Python SDK translates each non-2xx response into a typed exception subclass of VeepError.

{ "error": "Invalid API key" }
StatusSDK exceptionMeaning
400ValidationErrorMalformed request body or arguments
401AuthErrorMissing or invalid API key
404CollectionNotFoundError / NotFoundErrorCollection or row does not exist
409CollectionAlreadyExistsError / FileAlreadyExistsErrorName conflict
422UploadErrorUpload payload could not be parsed (bad parquet, mismatched dimension, etc.)
425 / 503CollectionNotReadyError / ServerErrorCollection is still distributing or workers are unavailable
5xxServerErrorUnexpected server-side failure

The SDK exposes the full hierarchy under veep.exceptions:

from veep.exceptions import (
    VeepError,                       # base class for everything below
    AuthError,
    ValidationError,
    NotFoundError, CollectionNotFoundError,
    CollectionNotReadyError,
    CollectionAlreadyExistsError,
    UploadError, FileAlreadyExistsError,
    QueryError,
    TimeoutError,
    ServerError,
)