The drift window: why your RAG retrieves stale chunks

A support ticket landed on a Tuesday. A customer’s internal assistant was telling employees the wrong on-call rotation. The runbook in the source-of-truth wiki was correct. The bot’s answer was confidently, specifically wrong — naming an engineer who had rotated off two weeks earlier.

The wiki page had been edited eight times since that engineer rotated off. The retrieval still pulled the chunk from before the first edit.

This is the canonical shape of the drift window bug. The anchor post for this pillar (RAG without a second database) defines the window. This post is about how it actually breaks production, why your monitoring probably won’t catch it, and what specifically you have to change in the write path to make it impossible.

What the customer saw

The on-call rotation page in the wiki had this history:

T-21d  Page created. On-call: Anita.
T-14d  Anita rotates off. Page edited: On-call: Brian.
T-12d  Brian's title gets fixed. Page edited.
T-09d  Holiday coverage added. Page edited.
T-02d  Phone numbers updated. Page edited.
T-00   Employee asks the bot: "who's on call this week?"
T-00   Bot answers: "Anita is on call."

Every edit since T-14d enqueued a re-embed job. Five jobs. The queue worker had been wedged for ten days behind a poison-pill row that nobody had alerted on. The vector store still held the embedding of the T-21d revision. The retrieval was precisely correct — the chunk it returned was the best match for the query, given what was in the index. The chunk was just twenty-one days out of date.

The bot did not hallucinate. The pipeline returned stale ground truth.

Why monitoring missed it

The team had monitoring. It looked roughly like this:

• Queue depth alert: pages at >10k pending jobs.
• Worker error rate: pages at >1% of jobs erroring.
• Embedding API latency: pages on P99 >3s.
• Vector store write latency: pages on P99 >500ms.

None of these fired. The queue depth was steady — five jobs sat behind the poison pill, and the rest of the system kept up. The worker wasn’t erroring; it was waiting. The embedding API was fine. The vector store was fine.

What was missing is the one metric that actually corresponds to the bug: the time between a source row’s updated_at and the time its embedding row was last written. Call it drift_age_seconds. Without that metric, a queue can be “healthy” by every other measure while individual rows are silently weeks stale.

Most monitoring stacks don’t compute this because computing it requires joining the source store to the vector store, which usually live in different databases. So the metric gets postponed. Then the bug ships.

Why retries don’t save you

The reflexive fix to a drift-window incident is to make the pipeline more robust:

1. Better retry logic on the worker.
2. Idempotency keys on embedding requests.
3. A separate “stale-row scanner” that periodically diffs source and vector store and re-enqueues.
4. A health-check job that picks a random row and verifies the embedding is current.

These help. They do not close the window. They shorten it, on average, while leaving the tail unchanged. A poison pill, a deploy gone sideways, a partial outage of the embedding provider — any of these and the window reopens for whichever rows get stuck behind the failure. The customer that hits one of those rows during the window sees the bug.

The window exists because the source-of-truth write and the embedding write are two writes to two systems. Any architecture that keeps that property has a drift window. Architectures that collapse it into one write do not.

The fix: one transaction, both columns

The anchor post walks through the schema. The short version: the embedding column lives on the same row as the source text. The write that updates the text is the write that updates the embedding. No queue, no worker, no separate store.

The wiki-page table looks like this:

CREATE TABLE wiki_pages (
  id            UUID PRIMARY KEY,
  slug          TEXT UNIQUE NOT NULL,
  body          TEXT NOT NULL,
  body_tsv      tsvector GENERATED ALWAYS AS (to_tsvector('english', body)) STORED,
  embedding     vector(1536),
  embed_model   TEXT,
  embedded_at   TIMESTAMPTZ,
  updated_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

CREATE INDEX ON wiki_pages USING hnsw (embedding vector_cosine_ops);

The edit endpoint runs:

BEGIN;

  UPDATE wiki_pages
     SET body = $1,
         updated_at = now()
   WHERE slug = $2
  RETURNING id, body;

  -- application embeds $body inside the same request
  UPDATE wiki_pages
     SET embedding   = $3,
         embed_model = $4,
         embedded_at = now()
   WHERE id = $5
     AND updated_at = $6;  -- CAS guard: did anyone else win the race?

COMMIT;

The embedding call happens between the two UPDATE statements, inside the same transaction. The transaction commits only if both writes commit. The retrieval index sees body and embedding change atomically.

There is no queue. There is no worker. There is no second store. The drift window for this row is zero by construction.

What you give up

This is a tradeoff, not a free lunch. Three things change:

Write latency includes the embedding call. A wiki edit no longer returns in 30ms. It returns in 200-800ms, dominated by the embedding model. For an editor UI this is fine — the human just clicked save. For a hot write path (logging, telemetry, high-frequency events) it is unacceptable, and you do want a queue. The drift window is the price you pay for write-path simplicity; for high-frequency writes the price is wrong.

A failed embedding call fails the write. If the embedding provider is down, you cannot save the document. This is usually the wrong behavior. The pattern around it is to make the embedding nullable, let the write commit with embedding IS NULL, and have a backfill worker pick up the null rows. The retrieval query filters out null-embedding rows or falls back to text search for them. The window for those specific rows is non-zero, but it is bounded to “this row was just created” rather than “any row that’s ever been edited.”

Re-embedding the whole corpus is no longer a separate pipeline. Model upgrades require a migration, not just a worker config change. The anchor post covers the SKIP LOCKED pattern for this. It’s straightforward but it is a thing you now have to write.

How to know if you have this bug right now

Three checks you can run today against your existing two-store RAG pipeline:

1. Pick ten source rows that were edited in the last month. For each, fetch the source updated_at and the vector-store’s last-write timestamp for that row. Compute the gap. If any gap is more than a few minutes, you have a drift window in production right now.

2. Look at your worker’s oldest pending job age. Not the queue depth — the age of the oldest job. If it’s older than your error budget allows, your monitoring is alerting on the wrong dimension.

3. Search your incident history for the phrase “stale” or “outdated” in customer-reported bugs. Drift-window bugs almost always get reported as “the assistant gave me old information.” They rarely get attributed to the pipeline because the pipeline looks healthy.

If any of these turn up something, the drift window is your bug. The fix is in the schema, not the queue.

TL;DR

• The drift window is the gap between a source-row update and its embedding update.
• Standard pipeline monitoring (queue depth, error rate, latency) does not detect it. The metric you need is per-row drift_age_seconds, which most stacks don’t compute.
• Retries and stale-row scanners shorten the window but cannot close it; the window exists because the source write and the embedding write are two writes to two systems.
• Putting the embedding on the source row and updating both in one transaction collapses the window to zero, at the cost of write latency and an embedding-provider dependency on the write path.
• For editor-driven content, the tradeoff is almost always worth it. For high-frequency writes, keep the queue and accept the bounded window.

Related: RAG without a second database (anchor), Building an agent memory layer on RedDB (atomic episodic+semantic writes use the same pattern).