Best Embedding Models for RAG (2026): Qwen3, Voyage 4, BGE-M3, OpenAI and More

By Rajesh Kannan, Co-Founder & CTO, Gramosoft · Last updated: November 7, 2026 · Prices, licenses and specs checked on each vendor's own page, model card or repository on November 7, 2026

The embedding model decides which chunks of your documents your AI ever gets to read. Pick a weak one and a correct answer sitting in your knowledge base never reaches the prompt. Pick the wrong license and your legal team stops the launch. And switching later means re-embedding everything.

We build RAG systems and AI knowledge agents for businesses, so this guide starts with where Gramosoft fits. Then it ranks the embedding models worth testing in 2026, open-source and API, with prices, licenses and context limits taken from each vendor's own page and dated, plus how to test them on your own documents.

Where Gramosoft fits in your RAG pipeline

Gramosoft is a Chennai software company that builds AI products and custom software. We don't sell an embedding model, so we have no favourite here. We pick the model per project, after testing candidates on your own documents, and we build the parts around it that decide answer quality: parsing, chunking, search, reranking and access control.

Swipe the table sideways on mobile →

We also build GramoERP, help desk software and GramoCare, the kind of systems a RAG assistant often has to answer questions about.

"Most RAG projects that disappoint don't have an embedding problem. They have a parsing problem, a chunking problem or no evaluation at all. Fix those, then let a hundred of your own questions pick the model."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

Best embedding models for RAG: the short answer

No single embedding model wins every RAG project. The deciding factors are whether text may leave your servers, which languages you serve, how long your chunks are, whether you need keyword-style matching and what you can spend. Use this table as a starting shortlist, then confirm it on a sample of your own questions.

Swipe the table sideways on mobile →

The embedding models at a glance

The table lists the models worth testing in November 2026, with the facts that decide production fit: license, vector size, maximum input and price. Every figure comes from the vendor's own pricing page, model card or repository, checked on November 7, 2026. Dimensions are the default output; most models can be shortened.

Swipe the table sideways on mobile →

The best embedding models for RAG, ranked

Each entry covers what the model does well, who it suits and where it falls short. Benchmark numbers are the vendor's own model-card claims, labelled as such, because the public leaderboard uses several versions that can't be compared directly. We list Gramosoft first because many teams need a working pipeline more than a model.

1. Gramosoft: RAG systems built and tested on your documents

Best for: businesses that want a knowledge assistant or RAG feature that answers correctly from their own files, without hiring a retrieval team. We sample your documents and real questions, test three or four embedding models on them, then build parsing with GdoczAI, hybrid search, reranking and access control around the winner. GsearchAI brings the result into Teams, Slack or WhatsApp.

Where it falls short: if your engineers want to own and tune the retrieval code themselves, they still need to learn the stack we choose. We document the pipeline and the evaluation set and hand both over.

2. Qwen3-Embedding: the open-source default

Best for: self-hosted RAG where your legal team wants a permissive license. Alibaba's Qwen team ships it in 0.6B, 4B and 8B sizes, all with 32K-token inputs, adjustable vector sizes and task instructions on the query side. The weights are Apache 2.0, and a matching Qwen3-Reranker comes in the same three sizes.

Benchmark (model card): the 8B model scored 70.58 on MTEB Multilingual and ranked first on that leaderboard on June 5, 2025. The 0.6B scored 64.33 against 59.56 for BGE-M3 on the same table.

Where it falls short: the 8B model needs a GPU with roughly 16 GB for its weights alone in bf16 (our arithmetic: 8 billion parameters at 2 bytes each), and its 4,096-dimension vectors are heavy to store. Start with the 0.6B and move up only if your own test says so.

3. Voyage 4: the best hosted API

Best for: teams that want strong retrieval without running GPUs. The family has three tiers: voyage-4-large at $0.12, voyage-4 at $0.06 and voyage-4-lite at $0.02 per 1M tokens, each with 200 million free tokens and a 33% batch discount. All accept 32K tokens, and voyage-code-4 at $0.12 covers code search.

Why it stands out: Voyage says the 4-series models share one vector space, so you can index documents with voyage-4-large and embed queries with a cheaper tier without re-embedding.

Where it falls short: closed weights, so every chunk and query goes to Voyage's API. Anthropic does not make its own embedding model, and its embeddings documentation uses Voyage in its examples, which is why many Claude-based teams start here.

4. OpenAI text-embedding-3: the low-friction default

Best for: a working prototype today, especially if you already use OpenAI. text-embedding-3-small outputs 1,536 dimensions and text-embedding-3-large 3,072, both reducible, with 8,192-token inputs. OpenAI's embeddings guide prices them at about 62,500 and 9,615 pages per dollar at roughly 800 tokens a page, which works out to $0.02 and $0.13 per 1M tokens.

Benchmark (vendor): 62.3% (small) and 64.6% (large) on the original MTEB.

Where it falls short: these are the oldest models on this list, and their scores come from an older benchmark version than the newer open models report, so test before you assume either way.

5. Gemini Embedding 2: one index for text, images, audio and PDFs

Best for: knowledge bases that mix documents with images, short videos or audio. Google's model page lists text, image, video, audio and PDF inputs, 8,192 text tokens, and output from 128 to 3,072 dimensions (768, 1,536 and 3,072 recommended). It has been stable since April 2026 and covers more than 100 languages.

Where it falls short: 8,192 tokens is short next to the 32K models. Google's pricing page could not be loaded when we checked on November 7, 2026, so check the current per-token rate on the Gemini API pricing page before you budget.

6. Cohere Embed 5: 128K-token inputs and enterprise support

Best for: long contracts, slide decks and page images, and teams that want private deployment options. Launched on September 30, 2026, it comes as Embed 5 Pro at $0.12 and Embed 5 Fast at $0.08 per 1M text tokens, both $0.40 per 1M image tokens, with 128K-token context, 100+ languages and vectors from 256 to 2,048 dimensions.

Where it falls short: it is new and had no complete public benchmark score when we checked, so treat Cohere's comparisons as vendor claims. Pro and Fast share one vector space, so you can mix them.

7. BGE-M3: hybrid search from one model

Best for: corpora full of exact terms, product codes, chemical names or part numbers. BGE-M3 is MIT licensed, reads up to 8,192 tokens, works in 100+ languages and produces dense, sparse and multi-vector output in one pass. Its own model card recommends hybrid retrieval plus a reranker.

Where it falls short: it is older than the 32K-context models and has no complete score on the newest English leaderboard, so newer small models may beat it on long documents. Some runtimes, such as Ollama, return only its dense vectors.

8. Microsoft Harrier: the highest claimed open score

Best for: teams that want the top multilingual score under an MIT license, or a small model that punches above its size. The family comes in 270M, 0.6B and 27B sizes, all with 32,768-token inputs. Queries need a one-line task instruction; documents don't.

Benchmark (model card): 74.3 on MTEB Multilingual v2 for the 27B, 69.0 for the 0.6B and 66.5 for the 270M.

Where it falls short: the 27B needs about 54 GB for weights in bf16 and outputs 5,376-dimension vectors, so it is a quality ceiling, not a default. The 0.6B is the practical pick.

9. NVIDIA Nemotron-3-Embed-8B: built for retrieval

Best for: retrieval-heavy RAG over legal, finance or technical text. Released on July 16, 2026, it outputs 4,096-dimension vectors from inputs up to 32,768 tokens. Its card says it reached state of the art on the multilingual RTEB retrieval leaderboard on its release date and that it is ready for commercial use.

Where it falls short: it uses NVIDIA's OpenMDW-1.1 license rather than Apache or MIT, so have legal read it, and it needs a GPU like the other 8B models.

10. EmbeddingGemma 2: small, multimodal and Apache 2.0

Best for: on-device, edge or low-cost servers. Google's model has 740M parameters in total, one 768-dimension space for text, code, images, video and audio, 8,192-token inputs and an Apache 2.0 license. Its card reports 68.46 on MTEB English v2, 61.36 on Multilingual v2 and 78.68 on MTEB Code.

Where it falls short: run it in bfloat16 or float32; its card warns that float16 returns NaN or quietly degraded embeddings without an error.

Open weights don't always mean open license

Downloadable weights and commercial rights are different things, and this is where projects get stopped late. Several high-scoring embedding models carry non-commercial or custom licenses. Check the license of the exact checkpoint you download, because licenses can differ inside one model family and change between versions.

Swipe the table sideways on mobile →

How to read MTEB scores without being misled

The MTEB leaderboard is a useful first filter and a poor final judge. Scores come from different versions, English v1, English v2 and Multilingual v2, which can't be compared across rows. The overall average mixes retrieval with classification and clustering, and some top models trained on part of the test data.

• Compare like with like: a 74.3 on Multilingual v2 and a 64.6 on the original English benchmark are different scales.
• Sort by retrieval, not the average: for RAG, the retrieval column or the retrieval-only RTEB board matters most.
• Treat vendor numbers as claims: every score in this guide comes from a model card or vendor page, not our own test.
• Check coverage: many popular API models have no complete score on the newest boards, which says nothing about their quality.

What matters more than the embedding model

On a well-built pipeline, the gap between two strong embedders is often smaller than leaderboards suggest. The bigger levers sit around the model: how cleanly text comes out of your PDFs, how you split it, whether keyword search catches exact terms, and whether a reranker fixes the order before the language model reads anything.

• Parsing: two-column layouts, tables and formulas get mangled by basic PDF extraction, and no embedder can recover text that isn't there. Read 50 random extracted pages before you compare models. GdoczAI handles scanned and complex documents.
• Chunking: paragraph- or section-sized chunks usually retrieve better than one vector for a whole document, even when the model accepts 32K tokens. Keep headings inside each chunk.
• Hybrid search: keyword search (BM25) catches IDs, error codes and product names that vectors blur. Merge both result lists.
• Reranking: a cross-encoder reranker over the top 50 candidates often lifts the top five more than a better embedder does.
• Numbers and dates: embeddings treat "0.5 mol/L" and "5.0 mol/L", or 2024 and 2025, as nearly the same. Put them in metadata and filter on them.

What embeddings really cost: tokens, storage and re-indexing

API embedding prices look tiny, but three costs add up: embedding the corpus and every query, storing the vectors, and re-embedding everything whenever you change model, chunking or vector size. Self-hosting turns the token bill into a fixed GPU bill. The figures below are list-price and storage arithmetic, not benchmarks.

Swipe the table sideways on mobile →

Storage grows in a straight line with vector size. One million float32 vectors take about 3.1 GB at 768 dimensions, 4.1 GB at 1,024, 6.1 GB at 1,536, 12.3 GB at 3,072 and 16.4 GB at 4,096, before any index. At 1,024 dimensions, int8 cuts that to about 1 GB and binary to about 128 MB. Most models here support shorter vectors, so test 512 or 768 dimensions on your own questions before paying to store the full size.

How to test embedding models on your own documents

An afternoon of testing on your own data tells you more than any leaderboard. Build a small labelled set of real questions, each paired with the chunk that answers it, embed the same chunks with every candidate, and score how often the right chunk lands near the top. Keep corpus and chunking fixed across runs.

1. Collect 100 to 200 real questions from search logs, support tickets or the people who will use the assistant. Include exact codes, paraphrases, table lookups and a few with no answer.
2. Mark the answering chunk for each question.
3. Shortlist three or four models that pass your license, language and hosting rules, plus one cheap baseline.
4. Use each model's query format: Qwen3 and Harrier want a task instruction on queries, BGE-M3 needs none.
5. Score Recall@10 and MRR@10, then record speed, storage and price next to them.

This short script runs that comparison with Sentence Transformers. It expects a file with your chunks and labelled questions, and it prints one line per model.

eval_embeddings.py

# eval_embeddings.py: compare embedding models on your own labelled questions
# pip install sentence-transformers numpy
import json
import numpy as np
from sentence_transformers import SentenceTransformer

TASK = "Instruct: Given a question, retrieve the passage that answers it\nQuery: "

def evaluate(model_id, chunks, queries, k=10, query_prompt=None):
    """chunks: list of passages. queries: list of (question, index of the answering chunk)."""
    model = SentenceTransformer(model_id)
    docs = model.encode(chunks, normalize_embeddings=True, batch_size=32)
    qs = model.encode([q for q, _ in queries], normalize_embeddings=True, prompt=query_prompt)
    top = np.argsort(-(qs @ docs.T), axis=1)[:, :k]
    hits = [np.where(row == gold)[0] for row, (_, gold) in zip(top, queries)]
    recall = np.mean([len(h) > 0 for h in hits])
    mrr = np.mean([1 / (h[0] + 1) if len(h) else 0.0 for h in hits])
    return {"model": model_id, f"recall@{k}": round(float(recall), 3), f"mrr@{k}": round(float(mrr), 3)}

if __name__ == "__main__":
    # eval_set.json: {"chunks": ["...", ...], "queries": [["question", 12], ...]}
    data = json.load(open("eval_set.json"))
    queries = [tuple(q) for q in data["queries"]]
    for model_id, prompt in [("Qwen/Qwen3-Embedding-0.6B", TASK),
                             ("microsoft/harrier-oss-v1-0.6b", TASK),
                             ("BAAI/bge-m3", None)]:
        print(evaluate(model_id, data["chunks"], queries, query_prompt=prompt))

Run the winner again with a reranker added and with shorter vectors. If two models finish within a point or two, choose on license, price and hosting.

How to choose an embedding model for RAG

Start from your hard rules, not from the leaderboard. Whether data may leave your servers, which languages you serve and what license your lawyers accept remove more candidates than any benchmark. Then choose between hosted and self-hosted, and let your own labelled questions settle the last two or three options.

1. Data residency first: if text can't leave your network, choose open weights such as Qwen3-Embedding, Harrier or BGE-M3.
2. Check the license of the exact checkpoint.
3. Match context to your chunks, not your whole documents.
4. Add keyword search if users search for codes, names or numbers.
5. Pick the smallest model and vector size that clears your quality bar.

Switching embedding models later

Vectors from different models live in different spaces and can't be compared, so changing models means re-embedding every chunk and rebuilding the index. That is a batch job, not a disaster, if you plan for it from day one. A few model families also share one space across their sizes.

• Store raw text and chunk IDs separately from vectors, plus the model name and version beside each vector.
• Build the new index alongside the old one, compare on your evaluation set, then switch queries over.
• Shared spaces: Voyage 4 models are compatible with each other, and Cohere Embed 5 Pro and Fast share one space.
• Re-run your evaluation set every quarter or when a strong new model appears.

"Treat the embedding model as a part you will replace. Keep the raw text, the chunk IDs and the test questions, and a model change becomes an overnight job instead of a project."

Rajesh Kannan, Co-Founder & CTO at Gramosoft

Frequently asked questions

Related guides

• ChatGPT vs Claude vs Gemini for Business (2026): Which AI Should Your Company Use?
• 10 Best Claude Code Alternatives in 2026 (Free and Paid)
• Web Application Development Cost in Chennai (2026): What Drives the Price