Here's the part that sounds like a magic trick. You hand a computer the word dog, and it hands back a list of a few hundred numbers. Hand it puppy, and the numbers come back almost the same. Hand it invoice, and the numbers are completely different. Somehow the machine "knows" that dogs and puppies are related and invoices aren't — even though nobody ever wrote down that fact, and even though the words share no letters.
Nobody gave it a dictionary. Nobody wrote rules. So where does the knowledge come from?
The honest answer is more interesting than the magic-trick version, and you don't need any math to get it.
It learned the way you'd learn a language abroad
Imagine you move to a country whose language you don't speak. Nobody hands you a translation dictionary. But you hear the language all day, every day, for a year. You notice that a certain word shows up whenever people are around food. Another word appears whenever someone's leaving. You never get a definition — but by the end of the year you know what the words mean, because you saw the company they keep.
That's exactly how the machine learns, except its "year abroad" is reading a huge slice of the internet — Wikipedia, books, news, forums, code — billions of sentences. It never gets definitions either. It just gets one relentless exercise, over and over: given the words around a blank, guess the missing word.
"I took my ___ for a walk." Dog. Leash. Puppy. "Please pay the ___ by Friday." Invoice. Bill.
To get good at that game, the machine has no choice but to start grouping words by the situations they show up in. "Dog," "puppy," "leash," and "walk" keep appearing near each other, so it learns to treat them as related. "Invoice" and "bill" travel together in a totally different crowd. It's not understanding in any human sense — it's noticing patterns of company at a scale no person could. But the result behaves remarkably like understanding.
There's an old line in linguistics that captures it perfectly: you shall know a word by the company it keeps. That single idea is the whole engine.
Turning "company it keeps" into numbers
Once the machine has learned which words hang out together, it needs a way to store that knowledge — and it stores it as position.
Picture a giant map. Every word gets a spot on it. The machine arranges the map so that words which keep similar company sit close together, and words from different worlds sit far apart. "Dog" and "puppy" end up as next-door neighbors. "Invoice" is clear across town. The map isn't drawn on paper with two directions — it has hundreds of directions at once, which is impossible to picture but easy to state: nearby means similar.
Those few hundred numbers the computer handed back for dog? They're just the coordinates of its spot on that map. That's all an embedding is — an address. The reason dog and puppy come back with nearly-identical numbers is that they're nearly in the same place, because in billions of sentences they kept nearly the same company.
The party trick that proves it's really "meaning"
If this were just "similar words get similar numbers," it'd be neat but shallow. What convinced a lot of skeptics is that the map turns out to have structure — you can do arithmetic on it.
Take the coordinates for king, subtract the coordinates for man, add the coordinates for woman. Where do you land? Almost exactly on queen.
Nobody built that in. It falls out on its own, because in all that reading, "king" related to "man" in the very same way "queen" related to "woman" — royalty, plus a gender — and the machine, arranging its map to honor every pattern it saw, ended up placing the words so that the relationship became a direction you can travel. Do the same trick with countries and capitals and you glide from Paris to France to Tokyo to Japan. The map didn't just memorize words; it captured the relationships between them.
That's the moment "it's just counting" stops feeling like a good description, even though it's still technically true.
Why this is the thing that makes vector search work
Everything a vector database does rests on this one idea. Because meaning got turned into position, "find things that mean something similar" becomes "find things that are nearby on the map" — and finding nearby points is something a computer can do fast, even across millions of items. Without the map, "similar in meaning" is a philosophy question. With it, it's a geometry question.
And here's the practical payoff: you don't have to build the map yourself, and you don't have to understand how it was trained to use it. Someone already spent enormous effort teaching a model the company that words keep. You hand that model your text, it hands you back coordinates, and you store them. The magic already happened; you just get to use it.
Closing
So the black box isn't really black — it's just unfamiliar. A machine read more text than any human ever could, played "guess the missing word" a trillion times, and in the process was forced to arrange words by the company they keep. Meaning became distance on a map. That's the whole secret.
Next, if you want the practical version — what these numbers look like, how many there are, and how to pick which model draws your map — read Vector embeddings 101. And the same trick works far beyond words: the same idea maps photos, sound, and video into the same kind of space.