Embeddings Deep Dive: Understanding Vector Representations

Turning words into math.

The Magic of `[0.1, -0.5, 0.8...]`

To a computer, the words Dog and Canine are just ASCII strings. They look nothing alike. D-o-g shares no letters with C-a-n-i-n-e.

To make an AI understand that they are related, we convert them into Embeddings: lists of floating-point numbers (vectors).

In this vector space:

King - Man + Woman ≈ Queen
Dog is mathematically close to Canine.
Apple (fruit) is far from Apple (tech company) if the context differs.

Choosing an Embedding Model

Not all vectors are created equal.

1. OpenAI `text-embedding-3-small` / `large`

Pros: Easy, cheap, decent performance. Multilingual.
Cons: You send your data to OpenAI. Dimensions are fixed.

2. Open Source (Hugging Face MTEB Leaderboard)

BGE-M3: Currently a beast for multilingual retrieval.
E5-Mistral: State-of-the-art but heavy.
Pros: Free, private (run locally), often beats OpenAI on benchmarks.

Dimensions Matter

Small (384 dims): Fast search, less nuance. Good for simple FAQs.
Large (3072 dims): Slow search, high nuance. Good for complex legal discovery.

The Dense Passage Retrieval Trap

Embeddings are Dense. They capture meaning. But they sometimes fail on Exact Matches.

Query: Part number XJ-998
Embedding match: Component YK-112 (Because they look like similar part numbers).

This is why purely vector search is dangerous for technical catalogs. You need Hybrid Search (Vectors + Keywords).

Conclusion

Your RAG system is only as good as your embeddings. If you are building a medical bot, use a medical embedding model (BioBERT etc.). If you are building a general bot, OpenAI is fine. But know the trade-offs.

Next in the series: [Production RAG Architecture](/blog/production-rag-architecture)

How This Article Was Made

This article is a live example of the AI-enabled content workflow we build for clients.

Stage	Who	What
Research	Claude Opus 4.5	Analyzed current industry data, studies, and expert sources
Curation	Tom Hundley	Directed focus, validated relevance, ensured strategic alignment
Drafting	Claude Opus 4.5	Synthesized research into structured narrative
Fact-Check	Human + AI	All statistics linked to original sources below
Editorial	Tom Hundley	Final review for accuracy, tone, and value

The result: Research-backed content in a fraction of the time, with full transparency and human accountability.

Why We Work This Way

Were an AI enablement company. It would be strange if we didnt use AI to create content. But more importantly, we believe the future of professional content isnt AI vs. Human—its AI amplifying human expertise.

Every article we publish demonstrates the same workflow we help clients implement: AI handles the heavy lifting of research and drafting, humans provide direction, judgment, and accountability.

Want to build this capability for your team? Lets talk about AI enablement →