TiDB Vector Search for RAG: When to Skip the Second DB

TiDB vector search earns its keep in one specific scenario: when a query needs to filter on live, frequently-changing relational data and rank by vector similarity in the same transaction. If that’s your actual access pattern, and it is for most multi-tenant RAG and agent memory systems, storing embeddings in TiDB alongside the data they describe removes a synchronization pipeline you’d otherwise be maintaining between a transactional store and a separate vector database. It is not a general argument for replacing a dedicated vector database in every case, and the architecture comes with a specific dependency and a specific query-planning risk that most teams don’t discover until a filtered query in production quietly stops using the index they built.

What “One Database” Actually Buys You

The mechanical reason this works is the same HTAP split that makes TiDB’s broader architecture possible. TiKV holds the row-based transactional data, TiFlash carries a columnar replica with the HNSW vector index built on top of it, and TiDB’s SQL layer can push a filter, a join, and a vector distance ranking down into a single query plan that resolves inside one ACID transaction boundary. The practical value is that an application never has to stitch together an answer from two systems with two separate consistency models: a dedicated vector store can tell you which items are similar, but it can’t join that answer against your live transactional data without crossing a system boundary, and that boundary is exactly where staleness and sync bugs accumulate in a typical RAG stack. Moonshot AI’s Kimi agent platform runs its persistent agent infrastructure on this model specifically so agents never have to reason across separate database boundaries for memory and retrieval.

The Dependency You’re Inheriting

Vector search in TiDB isn’t an add-on to the row store, it’s built on TiFlash, and that dependency has sharp edges worth knowing before you commit to the architecture. If a table doesn’t already have a TiFlash replica when you add a vector index, you have to create one manually first, and building a vector index on a TiFlash node with encryption at rest enabled isn’t supported at all, which is the kind of constraint that surfaces during a security review rather than during initial development. The index itself has real restrictions too: it can only be built on a single vector column, it can’t be combined with other columns into a composite index, and the column has to be a fixed dimension.

The operational discipline this demands is the same discipline you’d apply to any asynchronous replica, treated as its own monitored signal rather than assumed to be healthy. After a large ingest or backfill, checking INFORMATION_SCHEMA.TIFLASH_INDEXES to confirm the index actually finished building is not optional housekeeping, it’s the equivalent of checking replication lag on any other replica, and teams that skip it discover the gap only when a query that should hit the HNSW index falls back to something slower under load.

The Filtered-Query Trap Most RAG Teams Don’t Test For

Here is the detail worth treating as a required pre-production test, not an edge case: a highly selective WHERE clause can push TiDB’s query planner toward a table scan instead of the vector index. That matters enormously for RAG specifically, because the dominant real-world access pattern, filter to one tenant’s documents, then rank the remaining rows by embedding distance, is precisely the shape of query that triggers this behavior. A benchmark run without a realistic filter will look fine and tell you nothing about how the same query performs once it’s scoped down to a single tenant’s slice of a much larger table. The engineering judgment here is specific: test retrieval latency against your actual filter cardinality in a multi-tenant table, not against an unfiltered full-corpus benchmark, because the two can produce completely different query plans and completely different latency profiles for what looks like the same logical query.

When to Choose TiDB Vector Search, and When Not To

Choose TiDB’s approach when your retrieval queries genuinely need to filter against live relational state and rank by similarity in the same consistency boundary, multi-tenant document retrieval, agent memory scoped to a user or session, anything where a stale join between two separately-maintained systems would be a real correctness problem, not just an inconvenience. That’s the case where eliminating the sync pipeline between a transactional store and a vector store pays for itself immediately. Choose a dedicated, purpose-built vector database instead when your workload is large-scale, vector-only similarity search without a meaningful relational filtering requirement, or when you need index-type flexibility TiDB doesn’t yet offer. TiDB currently ships HNSW as its only vector index type, with alternatives like IVF and DiskANN still under evaluation, so workloads that specifically need those tradeoffs aren’t served by TiDB today regardless of how well the HTAP story fits otherwise.

At SIRAYA, the guidance for teams evaluating this has been consistent: don’t decide based on the architecture diagram, decide based on whether your actual query pattern filters before it ranks, and whether that filter is selective enough in production to be the thing that quietly takes your vector index out of the query plan. Test that specific case before you commit, because it’s the one failure mode a generic vector-search demo will never show you.