Embedding models: the component that turns sentences into vectors and determines search quality

Why embeddings were needed

Keyword search finds a document only when the query and the document share the same words. A search for "how to use annual leave" may miss a document titled "vacation request procedure" even though they mean the same thing. Keyword ranking functions such as BM25 are fast and precise, but they cannot tell that two texts mean the same thing. What computers do well is compare numbers. This led to the idea of turning sentences into lists of numbers in a way that gives sentences with similar meanings similar numbers. The result of this conversion is an embedding, and the model that performs it is an embedding model.

Today, embedding models handle the "retrieval step" in RAG (retrieval-augmented generation), chatbots and semantic search. No matter how capable an LLM is, it gives wrong answers if it is handed the wrong documents. A study proposing a framework for selecting embedding models paper names the choice of embedding model as one of the decisions with the greatest impact on results when building a search system. Yet it points out that it is also one of the least scrutinized decisions.

Embeddings are not used only for search. ADEPT paper rewrote raw data such as time series as text and represented it with text embeddings. The goal was to automate the data engineering stages of data collection, preprocessing and feature engineering. The study reported better predictive performance than existing benchmarks on healthcare, finance and industrial IoT datasets, even with data containing missing values or broken formats. This shows that the ability to turn sentences into vectors is becoming a general-purpose component.

Core idea: plotting coordinates on a map of meaning

An embedding model is like a surveyor drawing a huge map. It assigns a coordinate to every sentence, and the closer two sentences are in meaning, the closer they are placed on the map. But while a real map has only two axes, width and height, the map of embeddings is a space with a very large number of axes. How similar two sentences are is measured by how close their coordinates are. Cosine similarity, which measures the angle between two vectors, is commonly used.

The key design choice is to compress one sentence into one vector. This makes it possible to convert millions of documents into vectors ahead of time and store them. When a query comes in, only the query needs to be converted into a vector before searching for nearby coordinates, so it is fast. The cost is that information is lost in compression. This is why "2% growth" and "20% growth" can end up in almost the same place.

This creates a trade-off with other approaches. Late interaction methods in the ColBERT family keep a separate vector for each token and match queries and documents token by token, preserving more detail. In exchange, they require more storage and computation. In a study on Amharic retrieval paper, a ColBERT-based model recorded the highest MRR@10 (0.843) among the models evaluated. Single-vector models gain speed and simplicity, while token-level models gain precision. Adding a reranker (a model that reorders results) at the end is another way to bridge the same trade-off.

How it works

A sentence becomes a vector in four steps. First, a tokenizer splits the sentence into tokens (words or word pieces). Second, a transformer creates a representation for each token that reflects its surrounding context. Third, a pooling step combines the per-token vectors into one. Averaging is common, and sometimes the vector of a specific token is used as the representative. Fourth, the resulting single vector becomes the sentence's coordinate.

Models learn "similarity" through contrastive learning. They are trained to pull related pairs, such as a query and its correct document, closer together and push unrelated pairs apart. The triplet training used in an entity matching study paper groups three items together: an anchor record, a record of the same entity, and a record of a similar but different entity. The model is trained so that the correct pair is closer than the incorrect pair by at least a set gap (margin). In the end, what the model calls "similar" is a standard set by its training data. This point is the key to understanding the limitations and custom training discussed later.

In a search system, the flow looks like this. Documents are first cut into suitably sized pieces through chunking (document splitting), and each chunk is embedded and stored in a vector DB. When a query comes in, it must be converted into a vector with the same embedding model. Then ANN search (approximate nearest neighbor search) quickly narrows down candidate chunks that are close, and if needed, a reranker reorders them. The selection framework study paper advises against treating model selection in isolation. It should be seen as one decision within a pipeline that spans embedding generation, indexing and large-scale search, and chunking strategy. In cases like chatbots, where a conversation continues, retrieval sometimes needs to include previous messages. Hakim paper made this kind of conversation-history-aware retrieval a design goal.

How the field has developed: from general-purpose models to custom models

The starting-point studies for this material are less a single line of work than different angles on the same question: "Is a general-purpose embedding model enough, or does it need to be adapted?" The first axis is language. Hakim paper built three new training datasets for Persian, a language neglected in large-scale embedding research. It reported 8.5% higher performance than existing approaches on the FaMTEB benchmark. The Amharic retrieval study paper shows this effect even more clearly. An Amharic-specific model with 110 million (110M) parameters outperformed Arctic Embed 2.0, a multilingual model with 568 million (568M) parameters, by 17.6% (relative) on MRR@10 and 9.86% on Recall@10. A smaller 42M variant also stayed competitive while being more than 13 times smaller.

The second axis is the definition of "similar." The entity matching study paper considers general-purpose embeddings strong at semantic similarity. But it points out that they are not tuned to tell whether records refer to the same company or the same person. A difference of a few characters between records can mean the same entity or a completely different one. The study did triplet fine-tuning on synthetic data and reported a large improvement in distinguishing true matches from very similar non-matches.

The third axis is how to choose. The selection framework study paper compared commercial API models and several open-source models on English retrieval tasks. It concluded that the top model on a leaderboard is rarely the best for a specific deployment environment. It therefore offers practical criteria for choosing based on task, latency, cost and deployment constraints. In short, the common conclusion of these studies is that a general-purpose model is only a starting point, and it needs to be adapted when the language, the domain or the definition of "same" differs.

Where things are heading

There is agreement that adaptation matters, but the problem is cost. Custom training requires many pairs of the form "this document answers this query." Having people design such data is expensive. SYNTRA paper tackles this bottleneck directly. Given a collection of unlabeled documents and a few query-document examples, an agent infers the information needs and relevance criteria hidden in them. It then refines its own query-generation and relevance-judgment instructions using feedback on trial outputs. Using the same pipeline, it produced usable training data across 24 datasets in 15 domains. On 4 of the 6 tasks evaluated, it achieved better retrieval performance than human-calibrated data. On MS MARCO, the authors reported that performance kept improving as the amount of synthetic data was scaled across a 32-fold range.

Another direction is "changing without retraining." Recent embedding models are sometimes used with an instruction placed before the query, such as "Find the legal provisions relevant to this question." But contrastive learning alone often fails to make models follow instructions well. Attention Relay paper proposed passing the attention weights of an LLM that already follows instructions well directly into the embedding model's attention. No additional training is involved. Most combinations of 6 LLMs from the Qwen3, Llama 3.1 and OLMo 3 families and 10 embedding models came to reflect the instructions. The authors report that the aspects asked about in the instruction became prominent in the embedding, and that aspects previously diluted by averaging were recovered.

Both directions come from the same concern. The definition of "similar" needs to be adjusted for each task, and both aim to reduce the cost of doing so. SYNTRA approaches this by making data cheaply, and Attention Relay by skipping training altogether.

Limitations and open issues

First, a single odd token can shake the results. A study on sticky tokens paper found tokens that, when inserted repeatedly into a sentence, pull similarity toward a particular value. It found 868 such tokens in total across 40 checkpoints from 14 model families. These tokens mostly came from special or unused vocabulary entries and from fragmented word pieces in multilingual corpora. Larger models or vocabularies did not produce fewer of them. In downstream tasks such as clustering and retrieval, performance dropped by up to 50%. Systems that index external documents as-is need to pay attention to this.

Second, embedding models are weak with numbers. A study on numeracy paper evaluated 13 widely used embedding models on synthetic data in a financial context. The models generally failed to capture numerical details accurately. "Market share grew by 2%" and "grew by 20%" mean very different things, but they can look similar in embeddings. In fields where numbers matter, such as finance or healthcare, it is safer not to rely on embedding search alone and to add numeric filters or keyword conditions.

Third, results depend heavily on the model. A science mapping study paper embedded the titles and abstracts of 500,000 OpenAlex papers with 10 models and compared them. The overall layout, with 26 academic fields placed at their average positions, was similar across models. But when the 25 papers closest to a given paper within the same field and period were selected, two models overlapped on only 8 papers on average. Whether only titles or titles plus abstracts were used also changed the lists substantially. This means switching models actually changes the search results users see. So when replacing a model, all documents should be re-embedded and re-evaluated.

Common mistakes are also worth listing: choosing based on leaderboard scores alone, embedding documents and queries with different models, and confusing a general-purpose model's "similar" with the task's "same" (the same customer, the same contract). Blaming only the model without reviewing the chunking strategy is also common.

Choosing a model in practice and further study

Choosing a model follows an order. First, prepare a small evaluation set (Evals) built from your own data. Collecting real questions paired with the documents that answer them is enough. Next, compare several candidate models under the same chunking conditions. As the selection framework study paper points out, weigh not only accuracy but also latency, cost and deployment constraints (whether you can use an external API or must run the model in-house). For languages other than Korean or for specialized domains, a small specialized model can do better than a large general-purpose model. The Amharic case paper is an example. If performance falls short, consider fine-tuning; if training data is scarce, look at synthetic data approaches such as SYNTRA paper.

Rather than relying on embedding search alone, mixing in keyword search (hybrid search) and adding a reranker can make up for areas where embeddings are weak, such as numbers and proper nouns. This conclusion follows directly from the weaknesses covered in the earlier section on limitations.

To study further, start with the selection framework study paper. It walks through the entire retrieval pipeline in one pass, from generating embeddings to indexing and chunking. Then read the Amharic retrieval study paper, which shows the effect of adaptation in numbers, and the entity matching study paper, which covers the difference between 'similar' and 'the same'; together they give a sense of why adaptation matters. To understand the weaknesses, read the studies on sticky tokens paper and numeracy paper; for recent directions, continue with SYNTRA paper and Attention Relay paper.