Quickstart

From zero to searching 5,000 research papers in under a minute.

Why vector search?

Normal search finds documents that contain your exact words. Vector search finds documents that mean the same thing — even when the wording is completely different.

In this quickstart, you'll search for a paper titled "Are extrasolar oceans common throughout the Galaxy?" and find papers about super-Earths, exoplanet surveys, and microlensing — none of which share keywords with the query. The search works because each paper's meaning has been captured as a list of numbers (a vector), and similar meanings produce similar numbers.

Example 1: Find Similar Papers

~30 seconds

Install, sign in, upload a real dataset of 5,000 arxiv papers, then ask: "what's similar to this one?"

Install

# Install the SDK with parquet upload support (pulls pandas + pyarrow)
pip install "veep[pandas]"

# Install the LangChain wrapper plus an embedding library (Example 2)
pip install "langchain-vectorpanda" "langchain-huggingface" "veep[pandas]"

Download the dataset

5,000 arxiv paper abstracts, each with a pre-computed embedding vector. 12 MB download.

# Download 5,000 arxiv papers with pre-computed embedding vectors (12 MB)
# Source: https://huggingface.co/datasets/sondalex/arxiv-abstracts-2021-embeddings-10000
# Mirror: https://next.vectorpanda.com/data/quickstart-arxiv-minilm-5k/arxiv-abstract-minilm.parquet
import urllib.request
url = "https://huggingface.co/datasets/sondalex/arxiv-abstracts-2021-embeddings-10000/resolve/main/data/arxiv-abstract-minilm.parquet"
urllib.request.urlretrieve(url, "arxiv.parquet")

Sign in and upload to Vector Panda

# Import the Vector Panda SDK
from veep import VP

# Sign in. This opens your browser for Google or GitHub authentication.
# Your credentials are saved locally — you only do this once.
vp = VP.login()

# Create a collection called "arxiv". We tell Vector Panda which column
# contains unique paper IDs and which contains the embedding vectors.
vp.collections.create("arxiv", id_field="id", vector_field="embedding")

# Upload the dataset. Vector Panda ingests and indexes it in ~2 seconds.
vp.vectors.upsert("arxiv", "arxiv.parquet")

# Import veep for collection setup + the LangChain wrapper for queries
from veep import VP
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_vectorpanda import VectorPandaStore

# Sign in once — same browser flow regardless of which SDK you use next.
vp = VP.login()

# The pre-computed embeddings go through veep directly (LangChain's add_texts
# is text-first and would re-embed every row).
vp.collections.create("arxiv", id_field="id", vector_field="embedding")
vp.vectors.upsert("arxiv", "arxiv.parquet")

# Wrap the populated collection in a LangChain VectorStore for queries.
embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
store = VectorPandaStore(
    collection_name="arxiv",
    embedding=embeddings,
    client=vp,
)

Find similar papers

Our collection is live. Let's pick one paper and ask: "what else in this dataset is about the same topic?"

# Fetch a specific paper from the collection by its arxiv ID.
# This paper asks: "Are extrasolar oceans common throughout the Galaxy?"
sample = vp.vectors.fetch("arxiv", "0704.3024")

# Print what we're searching for
print("Searching for papers similar to:")
print("  'Are extrasolar oceans common throughout the Galaxy?'")
print()

# Use that paper's vector to find the 5 most similar papers.
# with_metadata=True returns each paper's title and abstract.
results = vp.vectors.query(
    "arxiv",
    sample.vector,
    top_k=5,
    with_metadata=True,
)

# Print each result: similarity score and paper title
for r in results:
    print(f"  {r.score:.3f}  {r.metadata['title'].strip()}")

# Fetch a specific paper through the underlying client.
sample = store.client.vectors.fetch("arxiv", "0704.3024")

print("Searching for papers similar to:")
print("  'Are extrasolar oceans common throughout the Galaxy?'")
print()

# Run the similarity search by vector. Returns LangChain Document objects.
results = store.similarity_search_by_vector_with_score(sample.vector, k=5)

for doc, score in results:
    print(f"  {score:.3f}  {doc.metadata['title'].strip()}")

Output:

Score	Paper
1.000	Are extrasolar oceans common throughout the Galaxy?
0.699	Ground-based Microlensing Surveys
0.648	The HARPS search for southern extra-solar planets XI. Super-Earths in a 3-planet system
0.635	Detailed Models of super-Earths: How well can we infer bulk properties?
0.588	Predicting the frequencies of diverse exo-planetary systems

The first result (1.000) is the paper itself — a perfect match. The next four are papers about exoplanets, super-Earths, and planetary surveys. No keywords in common with the query — vector search found them by the meaning of their abstracts.

Optional: check your collection status at any time with vp.collections.get("arxiv") to see vector count, dimension, storage, and serving status.

Example 2: Search in Plain English

~5 minutes

Example 1 searched using a vector from the dataset itself. Now let's type plain English and find matching papers. We'll use all-MiniLM-L6-v2 — the same model that created the dataset's embeddings — to convert our questions into vectors. Using the same model matters: it's like making sure you and the dataset are speaking the same language.

# Install sentence-transformers (converts text → vectors).
# Downloads a small model (~80 MB) the first time you run it.
pip install sentence-transformers

# Import the Vector Panda SDK
from veep import VP

# Import the sentence embedding library
from sentence_transformers import SentenceTransformer

# Reuse the credentials saved by VP.login() in Example 1.
# No browser needed — reads from ~/.veep/credentials.json
vp = VP.from_creds()

# Load the same embedding model that produced the dataset's vectors.
# Using the same model ensures our queries land in the same "space."
model = SentenceTransformer("all-MiniLM-L6-v2")

# Three questions — plain English, any topic you're curious about
questions = [
    "How do social networks change over time?",
    "How do astronomers search for Earth-like planets?",
    "What determines the structure and shape of galaxies?",
]

for q in questions:
    # Convert the question to a 384-dimensional vector
    query_vector = model.encode(q).tolist()

    # Search the arxiv collection for the 3 most similar papers
    results = vp.vectors.query(
        "arxiv",
        query_vector,
        top_k=3,
        with_metadata=True,
    )

    # Show the question and its matching papers
    print(f"\n« {q} »")
    for r in results:
        print(f"  {r.score:.3f}  {r.metadata['title'].strip()}")

# When you're done experimenting, delete the collection
vp.collections.delete("arxiv")

# LangChain's HuggingFaceEmbeddings wraps sentence-transformers.
# The store embeds the query string for you — no manual model.encode().
from veep import VP
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_vectorpanda import VectorPandaStore

embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
store = VectorPandaStore(
    collection_name="arxiv",
    embedding=embeddings,
    client=VP.from_creds(),
)

questions = [
    "How do social networks change over time?",
    "How do astronomers search for Earth-like planets?",
    "What determines the structure and shape of galaxies?",
]

for q in questions:
    results = store.similarity_search_with_score(q, k=3)
    print(f"\n« {q} »")
    for doc, score in results:
        print(f"  {score:.3f}  {doc.metadata['title'].strip()}")

# When you're done experimenting, delete the collection
store.client.collections.delete("arxiv")

Output:

« How do social networks change over time? »

Score	Paper
0.646	Quantifying social group evolution
0.534	Novelty and Collective Attention
0.504	Cascading Behavior in Large Blog Graphs

« How do astronomers search for Earth-like planets? »

Score	Paper
0.622	Detecting and Characterizing Planetary Systems with Transit Timing
0.614	Interpreting and predicting the yield of transit surveys: Giant planets in the OGLE fields
0.611	SIM PlanetQuest: The Most Promising Near-Term Technique to Detect, Find Masses of Extra-Solar Planets

« What determines the structure and shape of galaxies? »

Score	Paper
0.650	Structures in the Universe and Origin of Galaxies
0.636	The Isophotal Structure of Early-Type Galaxies in the SDSS
0.635	Large Scale Self-Similar Skeletal Structure of the Universe

What just happened? You typed three questions in plain English and found matching research papers from 5,000 abstracts. A question about social networks found papers on group evolution. A question about finding planets found papers on transit timing and exoplanet surveys. A question about galaxy shapes found papers on cosmic structure. No keyword matching — vector search finds papers by meaning.

What's next?

Got your own data? Swap in your file — Parquet, CSV, JSON, and several other formats work out of the box. Just include an ID column and an embedding column. Vector Panda auto-detects your schema and starts processing immediately.

Heads up: the vp.collections.delete("arxiv") at the end of Example 2 removes the demo collection. If you skip that line, the collection stays live, you can keep querying it, but it counts toward your storage. Your $5 monthly usage credit covers most prototypes; no card is required for the first 90 days.

Documentation · Pricing