Embedding model selection, quality first and price last

By John M. · Updated 15 August 2026 · 6 min read

Embedding a 25,000-document corpus costs $1.73. That is the whole bill, once, for the entire knowledge base.

The arithmetic: 25,000 documents averaging 3,000 tokens is 75M tokens, and chunking with 15% overlap takes it to roughly 86.25M tokens. On OpenAI's text-embedding-3-small at $0.02 per million, 86.25 x $0.02 = $1.73. Choose the most expensive model in our snapshot instead, text-embedding-3-large at $0.13 per million, and it costs 86.25 x $0.13 = $11.21. The entire decision, across every credible option on the market, is worth $9.48.

Now put that next to what the system it feeds will spend. If the RAG application answers 50,000 questions a month with six retrieved chunks of 600 tokens plus the question and system prompt, that is about 4,500 input and 500 output tokens per answer. On Claude Sonnet 5 at $2.00/$10.00 the generation costs 4,500 x $2.00/M + 500 x $10.00/M = $0.014 per query, so 50,000 x $0.014 = $700 a month. Embedding the queries themselves adds 50,000 x 40 tokens = 2M tokens, which is $0.04 a month on 3-small and $0.26 on 3-large.

So the one-off corpus embedding is a quarter of one percent of a single month's generation bill, and the query-time embedding cost rounds to nothing at any price on the card. This has a straightforward implication that a surprising number of teams get backwards: do not select an embedding model on price. Select it on retrieval quality and dimension count, because the price difference is noise against your generation bill and the quality difference is not.

That $9.48 spread scales linearly, so it is worth knowing where the advice reverses. At a hundred times this corpus, 2.5M documents and roughly 8.6 billion tokens, the same choice is $172.50 against $1,121, and the gap is $948 rather than $9.48. Churn matters too: a wiki that rewrites 5% of itself every month re-embeds 4.3M tokens, which is $0.09 a month on 3-small and still irrelevant, but the same 5% on the larger corpus is $8.60 against $56.05 recurring. Below a few million documents, treat embedding price as noise. Above that, or with heavy churn and a high-dimension model, do the multiplication before you commit.

Dimensions are the cost that recurs

The embedding call is a one-off. The vector it returns is a monthly bill for as long as the product exists, and its size is set by the dimension count.

At 600 tokens per chunk, that 86.25M-token corpus is about 143,750 vectors. text-embedding-3-small returns 1,536 dimensions, which at four bytes per float is 6,144 bytes per vector: 143,750 x 6,144 = 883 MB. text-embedding-3-large returns 3,072 dimensions and therefore 1.77 GB for exactly the same content. For a low-latency approximate-nearest-neighbour index that wants to be resident in memory, that difference is a bigger instance every month, forever, in exchange for a $9.48 saving once. It also shows up in query latency, because distance computation scales linearly with dimensions and you are doing a great many of them per search.

Two levers fix this without going back to the cheaper model. Truncation is the first: 3-large supports a dimensions parameter, so you can keep the stronger model and store 1,024-dimension vectors, cutting the index to 589 MB while retaining most of the retrieval quality. The second is quantisation. Storing 1,536-dimension vectors as int8 rather than float32 takes the index from 883 MB to about 221 MB, and on most corpora the recall loss is small enough to be recovered by a reranker. Between them, dimension count is negotiable in a way that price is not, which is another reason to stop optimising the wrong variable.

Changing your mind costs a migration, not a fee

The other downstream cost is that embeddings are not portable. Vectors from two different models are not comparable, so switching means re-embedding the entire corpus. The API charge for that is the same $1.73 to $11.21, which is nothing. The migration is not nothing.

You will run the full pipeline again, which for a corpus of this size is hours of wall-clock time and whatever your chunking and extraction stack costs to operate. You will hold two indexes concurrently while you validate, so plan for double the storage during the cutover. You will need to backfill every document that arrived while the migration was running, which means the ingestion path has to write to both indexes for the duration.

Then you recalibrate, and this is the part that catches people. Similarity thresholds are model-specific. A cosine cut-off of 0.78 tuned on 3-small carries no meaning on voyage-4, because the score distributions are different, and every downstream rule keyed to that number, the reranker cut-off, the "no relevant results" fallback, the top-k you settled on after a fortnight of tuning, has to be re-derived. Budget a week, not an afternoon, and make the choice properly the first time.

When voyage-code-4 earns its 6x premium

voyage-code-4 at $0.12 per million costs six times text-embedding-3-small at $0.02. On the corpus above that is $10.35 against $1.73, a difference of $8.62, once.

General-purpose embedding models are trained overwhelmingly on prose, and code retrieval breaks their assumptions in specific ways. Identifiers fragment badly under general tokenisation, so parseUserSessionToken does not reliably land near a query about session parsing. Two functions can be textually near-identical and semantically opposite, which prose models are not trained to distinguish. Most importantly, code search is asymmetric: the query is English and the target is not, and a model trained on code and docstring pairs handles that mismatch in a way a prose model does not.

The payback calculation is not close. Take a code assistant answering 20,000 queries a month. If weaker retrieval makes the agent open three extra files per query at 1,500 tokens each, that is 4,500 extra input tokens: 4,500 x $2.00/M on Claude Sonnet 5 is $0.009 a query, or 20,000 x $0.009 = $180 a month in wasted context. The $8.62 premium is repaid in under two days, and that ignores the developer time lost to answers that cite the wrong file.

The condition is that your retrieval targets are actually code. If the corpus is mostly prose documentation that happens to discuss code, with a few snippets, a general model is the right call and the premium buys nothing. The same logic applies to any domain model: it earns its price when the thing being retrieved is structurally unlike prose, and not otherwise.

Just test it

Because the entire price range is under $10 on a corpus this size, the argument about which model to pick costs more in meeting time than the experiment costs to run. Embed your corpus with both candidates for $1.73 + $10.35 = $12.08, assemble 50 real queries with the documents that should be returned for each, and measure recall at 10 and mean reciprocal rank on your own data. Public leaderboard rank tells you little about a specific technical corpus in a specific vertical, and the test that does tell you costs the price of lunch.

Once you have picked, put the corpus size, chunk length and query volume into the RAG cost calculator. The embedding line will be the smallest number on the page, which is the point.

All figures in this guide come from the price index, last verified 15 August 2026. Every entry links to the vendor page it was read from — see the price index and the methodology.