RAG without a second database

Most RAG stacks treat the vector store as a sibling of the source-of-truth database. The text lives in Postgres or Mongo, the embeddings live in Pinecone or Qdrant or a pgvector table, and a job somewhere is responsible for keeping them in agreement. That job is where the bugs come from.

This post is about the specific class of failure that disappears when the embedding column lives on the row it describes — and about the schema, transactions, and queries that make that possible.

The two-store assumption nobody questions

Open any RAG tutorial published in the last two years. The architecture diagram has, with depressing consistency, three boxes: a document store, an embedding model, and a vector database. An arrow goes from the document store to the model. Another arrow goes from the model to the vector store. At query time, the application embeds the user’s question and asks the vector store for nearest neighbors.

The diagram is clean. The implementation is not. The arrows hide a small distributed system.

In production, the arrow from “document store” to “embedding model” is a queue. A worker reads new and changed rows from the source database — sometimes by polling a updated_at column, sometimes by tailing a CDC stream, sometimes by a trigger that writes to an outbox table. The worker calls the embedding model, then writes the resulting vector to the vector store, keyed by the source row’s primary key.

If anything in that pipeline lags, fails, retries, or is deployed mid-flight, the vector store and the document store now disagree about the world. A query lands in this window. The user sees the bug.

The drift window, named

Call this the drift window: the interval, measured per row, between the moment a document changes and the moment its embedding in the vector store reflects that change.

For a healthy pipeline running well, the drift window is short — seconds, sometimes minutes. For a pipeline under load, after a queue backlog, during a re-embed migration, or right after a deploy, the drift window is hours or worse. We have seen production drift windows in the tens of hours, undetected, because there was no alarm watching the gap.

What does the user experience inside the drift window?

• A support article was updated this morning to reflect a price change. The embedding still encodes the old paragraph. The user asks “how much does X cost?” Retrieval pulls the article (correctly), but the snippet the model is grounded on — the chunk the retriever scored highest — still semantically matches the old wording. The answer is wrong.
• A legal team redacted a name from a document. The text on disk is clean. The embedding was generated from the unredacted text. Vector similarity search on the old name still pulls that document into context. The model dutifully includes the redacted name in its answer.
• A document was deleted from the canonical store. The vector remained, because the deletion event was on a different topic and the consumer for that topic was down. Search returns a ghost — a result whose source no longer exists. Click-through returns 404 or, worse, returns the previous tenant’s row.

Each of these is a real outage we have seen on customer postmortems. Each is mechanical, not stochastic. Each disappears when there is no second store to drift against.

Schema: the embedding belongs on the row

The mental shift is small but consequential. Stop thinking of “the embedding for document 17” as a separate object that has to be kept in sync. Start thinking of it as one more column on document 17.

Here is the table:

CREATE TABLE documents (
  id          UUID PRIMARY KEY,
  tenant_id   UUID NOT NULL,
  title       TEXT NOT NULL,
  body        TEXT NOT NULL,
  updated_at  TIMESTAMPTZ NOT NULL DEFAULT now(),

  -- the embedding lives here, on the same row, in the same engine
  body_embedding  VECTOR(1024) NOT NULL,
  embedding_model TEXT NOT NULL,         -- which model produced it
  embedded_at     TIMESTAMPTZ NOT NULL   -- and when
);

CREATE INDEX documents_tenant_embedding_idx
  ON documents
  USING hnsw (body_embedding vector_cosine_ops)
  WHERE tenant_id IS NOT NULL;

Three things to notice.

The vector is NOT NULL. It is not an optional projection of the row; it is part of the row’s identity. A document without an embedding is not a row that exists yet. This is the same posture we take toward title or body: missing is not a state.

The model name and timestamp are on the row. When the embedding model changes — and it will — you do not need a separate audit table to know which rows still need re-embedding. The row tells you.

The HNSW index is co-located. It is built on the same table, in the same engine, by the same transaction that wrote the row. There is no separate index server, no separate replication lag for the index, no separate failover.

Transactions: write the text and the vector together

The producer code looks like one round trip, not three:

async function upsertDocument(input: DocumentInput): Promise<Document> {
  const embedding = await embedder.embed(input.body)

  const row = await reddb.tx(async (tx) => {
    return tx.upsert('documents', {
      id: input.id,
      tenant_id: input.tenantId,
      title: input.title,
      body: input.body,
      updated_at: new Date(),
      body_embedding: embedding,
      embedding_model: embedder.modelId,
      embedded_at: new Date(),
    })
  })

  return row
}

The embedding call is outside the transaction (it is the slow network hop). The write inside the transaction includes both the text and the vector. Either both are visible to the next reader, or neither is. The drift window is exactly zero.

Compare to the two-store version of the same operation, where you have to choose between:

• writing the text first, then the embedding, and living with the window in between; or
• writing the embedding first, then the text, and serving search results for a document that does not exist; or
• writing to an outbox table inside the transaction, then having a downstream worker fan it out, and inheriting all the failure modes you were trying to avoid.

None of these are good. The third is the most defensible and it is still bad — the moment the worker is behind, you are back inside a drift window.

Reads: one query, not two

The retrieval side simplifies in proportion.

SELECT id, title, body, updated_at,
       1 - (body_embedding <=> $1) AS similarity
FROM documents
WHERE tenant_id = $2
  AND deleted_at IS NULL
ORDER BY body_embedding <=> $1
LIMIT $3;

One query. Tenant scoping and soft-delete filtering happen inside the same query that does the nearest-neighbor search, because they are columns on the same row. No second round trip to filter, no application-side join between “vector hits” and “current document state.”

This is the line we keep coming back to: the two-store architecture forces the application to do a join the database could have done. The application is wrong about that join far more often than the database would be.

What chunking looks like when chunks are rows

Real documents do not embed cleanly as a single vector. You chunk them. The two-store world makes this a second table in the vector store with a foreign key back to the document — and a second drift window for the chunk-to-document relationship.

In the single-engine model, chunks are rows. They are children of the document:

CREATE TABLE document_chunks (
  id              UUID PRIMARY KEY,
  document_id     UUID NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
  tenant_id       UUID NOT NULL,
  ordinal         INT NOT NULL,
  text            TEXT NOT NULL,
  embedding       VECTOR(1024) NOT NULL,
  embedding_model TEXT NOT NULL,
  embedded_at     TIMESTAMPTZ NOT NULL
);

CREATE INDEX chunks_embedding_idx
  ON document_chunks
  USING hnsw (embedding vector_cosine_ops);

ON DELETE CASCADE carries the chunks with the parent. When a document is deleted, its chunks vanish. When a document is rewritten, the upsert path deletes the old chunks and writes the new ones inside the same transaction:

await reddb.tx(async (tx) => {
  await tx.upsert('documents', { /* ...as above... */ })
  await tx.delete('document_chunks', { document_id: input.id })
  for (const [i, chunk] of newChunks.entries()) {
    await tx.insert('document_chunks', {
      id: crypto.randomUUID(),
      document_id: input.id,
      tenant_id: input.tenantId,
      ordinal: i,
      text: chunk.text,
      embedding: chunk.embedding,
      embedding_model: embedder.modelId,
      embedded_at: new Date(),
    })
  }
})

A reader hitting the database mid-transaction sees either the old set of chunks or the new one. Never a mix. Never a half-rewritten document. The retriever cannot return a chunk whose source paragraph has already been removed, because the chunk and the paragraph commit together.

What about re-embedding when the model changes?

The model will change. New embedding releases are quarterly events. Two-store architectures handle this with a forklift: stand up a parallel vector store, re-embed everything, swap. The forklift is its own outage class.

In the single-engine model, re-embedding is a background job that reads rows where embedding_model != <current>, recomputes the vector, and writes it back. Each row’s update is a one-row transaction. Readers continue to use whichever model the row currently has. The transition is monotone: every commit moves one more row to the new model, no row is ever in an inconsistent state, and you can pause and resume the migration without thinking about it.

async function reEmbedBatch(modelId: string, batchSize = 100) {
  const stale = await reddb.query(
    `SELECT id, body FROM documents
       WHERE embedding_model != $1
       LIMIT $2 FOR UPDATE SKIP LOCKED`,
    [modelId, batchSize],
  )

  for (const row of stale) {
    const v = await embedder.embed(row.body)
    await reddb.update('documents', {
      where: { id: row.id },
      set: { body_embedding: v, embedding_model: modelId, embedded_at: new Date() },
    })
  }
}

FOR UPDATE SKIP LOCKED lets you fan this out across workers safely — each worker grabs a chunk of stale rows and updates them, no coordination needed.

What we did not solve

Some things are unchanged by collapsing the stores.

You still need a good embedding model. Co-locating the vector does not improve recall.

You still need to chunk. Naive whole-document embeddings perform worse than chunked ones for long-form material, and that is independent of where the vector lives.

You still need evaluation. A retrieval pipeline you do not measure is one you cannot improve. Evals belong on the same database — see our piece on storing eval datasets as rows — but the act of building them is yours.

You still need to think about cost. A VECTOR(1024) column is four kilobytes per row before quantization. At a hundred million rows, that is non-trivial storage. HNSW indexes have their own memory footprint. None of this is free; it is just no longer split across two budgets owned by two teams.

The one bug class we removed

We did not build a magic retrieval system. We removed an entire category of operational failure — the kind that lives in the seam between “document changed” and “vector reflects the change” — by removing the seam. There is no drift window because there is no second store to drift against. There is no reconciliation job because there is nothing to reconcile. There is no orphaned vector because deletion of a document deletes its embeddings inside the same transaction.

That is one bug class, not all of them. But it is the bug class that, in our experience, drives the most off-hours pages on production RAG systems. Trading a queue and a worker and a sidecar for a column on a row turns out to be a very good trade.