Open source vs. commercial embedding models: which should your Chinese enterprise knowledge base choose?

A month ago, we helped a securities brokerage optimize a RAG knowledge base. They'd loaded compliance documents, internal memos, and customer Q&As into a vector database using a well-known commercial embedding API. The retrieval accuracy looked strong in testing. Two weeks after going live, the trading desk started complaining: "When I search for beneficial certificates, it throws back a bunch of fund purchase documentation." We checked the query logs and found the problem wasn't in the RAG architecture. It was at the foundation, in the embedding model itself. It was trained mostly on Simplified Chinese text, and it was poor at handling Taiwan's Traditional Chinese financial terminology, full-width punctuation, and formal language conventions. All of these subtle but important distinctions, it hadn't learned.

The same pattern repeats throughout enterprises. For Chinese knowledge bases, the highest-scoring model on the benchmark isn't necessarily the one you should use.

Bottom line: three variables that determine your choice

An embedding model converts text into vectors, positioning semantically similar content close together in vector space. It's the bedrock of your entire RAG retrieval; pick the wrong one, and even the strongest LLM can't fix it downstream. In Chinese enterprise settings, three factors matter: actual retrieval performance with Traditional versus Simplified Chinese, whether you can deploy on-premises, and cost structure. Getting these three dimensions right makes model selection straightforward.

The first is a trap many people miss: Traditional-Simplified asymmetry. Most open-source Chinese embeddings train mainly on Simplified Chinese. Traditional Chinese text will process, but recall drops on domain terms, variant characters, and Taiwan-Hong Kong language patterns. If your knowledge base is pure Traditional Chinese, typical in Taiwan's financial services, healthcare, and compliance sectors, you must run offline evaluation on your actual documents. Don't rely on the official C-MTEB benchmark average.

The second is data residency. Financial and healthcare data often can't make an external API call. It won't pass security review. When that's the case, commercial API models are off the table, regardless of accuracy. You can only choose from open-source models that support on-premises deployment.

The third is cost, and the cost structure differs. Commercial APIs charge per token with zero operational burden; open-source self-hosting means upfront GPU expenses plus ongoing engineering overhead. Small volume? Pick the API. Large volume long-term? Self-hosting pays for itself faster and stays cheaper.

Chinese embedding model selection: the trade-offs at a glance

Below are the models we assess most frequently in real-world deployments, organized for the Chinese context:

Read this table with one important caveat: the "Traditional Chinese Retrieval" column shows relative rankings, not absolute performance. The same model might perform quite differently on your legal contracts versus your customer service conversations. We never pick from the table alone. We use it to eliminate obvious mismatches, then we always test the remaining two or three models on your actual data.

The real hidden cost of open source: operations, not license

Engineers often hold a misconception: open-source means free. The license is free, yes. But running BGE-M3 stably on your servers means handling GPU acquisition or rental, deploying inference services (vLLM, TEI, and others), fine-tuning batch throughput, and recalculating vectors when you update versions. All of that is labor cost hiding behind the word "free."

Conversely, the hidden cost of commercial APIs is recalculation expense. When you've indexed millions of documents using one API model, switching models means your entire vector index needs re-embedding. That's a significant token expense and operational risk at scale. Factor this long-term switching cost into your decision when selecting an API.

If you're handling under several hundred thousand documents and your data can leave your infrastructure, start with a commercial API to launch quickly. Don't jump straight to self-hosting. Invest your engineering bandwidth in retrieval strategy and data preparation; you'll see better returns. Once data can't leave your premises or volume climbs toward millions, self-hosted open-source plus one or two GPUs break even in two to three months. Plus you control the model. You can fine-tune it on Traditional Chinese domain language, something commercial APIs cannot offer.

Two Chinese-specific details worth remembering

One is chunking. It causes problems more often than switching models does. Chinese has no word-spacing delimiters. If you use the English method of splitting by token count, you'll routinely split a complete semantic unit, and no embedding, however strong, can recover from that. Make sure your chunking respects Chinese sentence and paragraph boundaries.

Two is Traditional-Simplified normalization. If your knowledge base contains both Traditional and Simplified documents, implement a Traditional-Simplified mapping or synonym expansion on the query side. Otherwise, someone searching for one variant won't find the other, and they'll think the system is broken. A few lines of code fix this, but the impact is substantial.

In practice, selecting an embedding for a Chinese enterprise knowledge base has no standard answer. You get the answer by testing against your own data. When we deploy RAG systems at customer sites, we build a few dozen Traditional Chinese evaluation queries, run two or three candidate models to generate real recall metrics, then decide between on-premises and API. Demo metrics don't determine the outcome. What counts is your team running searches daily and getting accurate results.