Embedding Models Compared: The Engine Behind Search and RAG

When we talk about large language models (LLMs) for text generation, attention often goes to the models that formulate answers. However, the true power behind semantic search and Retrieval-Augmented Generation (RAG) lies with a different type of model: the embedding model. This overview helps you understand what these models do, how they differ, and how to choose the right model for your AI project—especially for Dutch-language applications.

What is an embedding model and what exactly does it do?

Computers do not understand human language; they only understand numbers. An embedding model is a specialized neural network that translates text (ranging from a single word to entire documents) into a long sequence of numbers, also known as a vector. This process is called 'embedding'.

The revolutionary aspect of modern embedding models is that they capture the semantic meaning of the text in that sequence of numbers. Texts with similar meanings generate vectors that lie close to each other in a mathematical vector space. This allows a search system to understand that the query "rent a car" is closely related to a document about "leasing a vehicle", even if there is no overlapping keyword. This forms the backbone of semantic search and is indispensable when you want to search through company documents using RAG applications.

How do embedding models differ from each other?

Not every model is the same. When evaluating embedding models, there are several technical and commercial aspects to consider.

1. Dimensions

The 'length' of the sequence of numbers that a model outputs is called the number of dimensions. Popular models range from 384 dimensions (such as all-MiniLM-L6-v2) to 1536 or even 3072 dimensions (such as OpenAI's text-embedding-3-large).

The trade-off: More dimensions generally mean the model can capture subtler nuances in language. However, this comes at the expense of storage space in your vector database and requires more computing power when performing searches. Many modern models offer the ability to shorten ('truncate') the number of dimensions with only minimal loss of accuracy.

2. Context Window

The context length determines how much text you can feed into the model at once to generate a single vector. Some lightweight models can only handle 512 tokens (about 350 words). If you exceed this, the rest of the text is ignored or truncated. More advanced models have a context length of 8192 tokens or more, meaning they can process entire chapters at once. Understanding your context window is crucial for determining your chunking strategy (how large you make the pieces of text before storing them).

3. Multilingualism and Dutch Language Performance

Many open-source models are primarily trained on English data. If you feed such a model with Dutch texts, the embeddings will be of low quality. Models like multilingual-e5-large or commercial variants from OpenAI and Cohere are specifically trained on dozens of languages. A good multilingual model also places translations close to each other in the vector space: the vector for "hond" lies in almost the exact same place as the vector for "dog", enabling cross-lingual search.

4. Costs per Million Tokens (Order of Magnitude)

If you need to index millions of documents, costs play a major role. Commercial APIs typically charge per million (1M) tokens. Although prices fluctuate, the current market situation is roughly as follows:

  • Large/Premium models: ~$0.10 to $0.15 per 1M tokens (e.g., Cohere English/Multilingual v3, OpenAI text-embedding-3-large).
  • Fast/Small models: ~$0.01 to $0.02 per 1M tokens (e.g., OpenAI text-embedding-3-small).
  • Self-hosted / Open-source: No API costs per token, but you pay for your own computing power (servers/GPUs). Read more about the Total Cost of Ownership here.

5. License Models

Open-source models often have licenses such as MIT or Apache 2.0, which allow commercial use without restrictions. However, some 'open weights' models have restrictive licenses for large enterprises. Always check the license terms (for example, on Hugging Face) before deploying a model in a production environment.

Open-Source versus Closed (Commercial) Models

The choice between an API service and self-hosting an open-source model depends on your capabilities and data privacy requirements.

Closed models (via API):

  • Examples: OpenAI (text-embedding-3 series), Cohere (embed-multilingual-v3), Google (text-embedding-004).
  • Advantages: No infrastructure to manage, extremely scalable, very high quality out-of-the-box (especially in multilingualism).
  • Disadvantages: Your data leaves your servers (which can be a bottleneck in highly regulated industries), vendor lock-in.

Open-source models (Self-hosted):

  • Examples: BGE series (BAAI), E5 series (Microsoft), Nomic Embed.
  • Advantages: Full control over data, one-time infrastructure costs (cheaper at massive scale), you can fine-tune the models on specific domain jargon (e.g., legal or medical Dutch).
  • Disadvantages: Requires MLOps expertise to host reliably, dependent on own hardware or cloud GPUs.

How do you measure which model fits Dutch text?

Because embeddings are abstract sequences of numbers, it is difficult to evaluate them 'by eye'. In the AI community, the MTEB (Massive Text Embedding Benchmark) is used to objectively compare models. MTEB tests models on tasks such as classification, clustering, and semantic search.

However, the standard MTEB leaderboard focuses heavily on English. For Dutch-language projects, you need to look specifically at multilingual MTEB scores or specific Dutch datasets. Because generic benchmarks do not always reflect the reality of your specific domain, the best measurement method is to create your own 'Golden Dataset':

  1. Collect 100 representative search queries that your users type in.
  2. Manually link each search query to the 3 to 5 documents that contain the perfect answer.
  3. Test different embedding models and measure how often the correct documents appear in the top 5 results. This percentage (also called Hit Rate or NDCG@5) tells you exactly which model performs best for your Dutch texts.

Reindexing: What happens when you switch models?

One of the biggest misconceptions in AI development is that you can interchange vectors. That is not the case. A vector generated by OpenAI's text-embedding-ada-002 lives in a completely different mathematical space than a vector from text-embedding-3-small, or a vector from a BGE model.

If you decide to switch to another embedding model, or if your provider phases out a model, you must reprocess ("re-embed") your entire dataset with the new model.

This reindexing takes time and—for commercial models—money. It means you have to send all texts from your source system through the API or your local model again to generate new vectors, and then store them in your vector database. Therefore, pay close attention to your provider's policy regarding the lifespan of their models, as explained in our guide on model versions and deprecation.

Decision Tree: Which embedding model should I choose?

Follow this step-by-step process to make an informed choice for your project:

  1. Is your data allowed to leave your own servers?
    • No: You are limited to open-source. Choose a multilingual model like multilingual-e5-large or bge-m3 and host it locally.
    • Yes: Proceed to step 2.
  2. Are your documents primarily in Dutch or multilingual?
    • No (primarily English): You have the widest choice. Models like nomic-embed-text or OpenAI's text-embedding-3-small are extremely cost-effective.
    • Yes (Dutch/Multilingual): Explicitly choose a model that excels in multiple languages. Cohere's embed-multilingual-v3 and OpenAI's text-embedding-3-large currently score very high on Dutch text comprehension.
  3. How large are the individual text blocks (chunks)?
    • Small paragraphs (fewer than 512 tokens): Standard models like the E5 family will suffice.
    • Long documents or complex context: Choose models with a long context length, such as the newer models from OpenAI (up to 8192 tokens) or bge-m3 (which also has specific optimizations for long texts).
  4. What is your budget when scaling up?
    • Low budget, high volumes: Strongly consider setting up a smaller, efficient open-source model, or choose the cheapest API tier (such as text-embedding-3-small with shortened dimensions to also reduce database storage costs).
    • Quality is crucial (e.g., medical/legal advice): Choose the model with the highest dimensions and best multilingual MTEB scores, or invest in fine-tuning an open-source model for your domain.

In Conclusion

Choosing the right embedding model is one of the most fundamental decisions when building AI applications. A poor search environment leads to poor context, which directly results in mediocre LLM responses (garbage in, garbage out). Take the time to experiment with a small subset of your own Dutch data and weigh the cost per token against accuracy. Want to know more about the broader AI landscape and how generative models fit into it? Then check out our comprehensive article: how to choose the right AI model?