Retrieval that knows when to stop
An AI system that declines is worth more than one that guesses. This is how we built one that does, and why declining is also cheaper.
The problem
A retrieval system that answers from thin context sounds exactly like one that answers correctly. That is the whole difficulty. The failure is silent, it is confident, and by the time anyone notices, the answer has already been acted on.
So the design question is not how the model composes an answer. It is what happens when the corpus does not contain one.
The query path, and the gate
status = COMPLETED. If nothing clears the threshold, the Response Composer is never invoked and the API returns HTTP 200 with is_grounded: false. The system declines, and it declines cheaply, because a refusal costs one model call instead of two.Ingestion is a state machine, not a script
Fourteen stages, each with its own persisted status, its own error code and an explicit retry policy. A document can be inspected at any point and you can see exactly which stage it reached.
task_acks_late and a prefetch of 1, a soft limit of 1800s and a hard limit of 2100s. Transient failures retry three times at 10s, 20s and 40s; permanent ones stop and record a machine-readable code.OCR only when needed
Tesseract 5.5.3 with OpenCV preprocessing, invoked only for pages that have no text layer. Running it unconditionally would multiply ingestion cost for no gain.
Chunking that respects pages
1000 characters with 150 of overlap, and a chunk never spans a page boundary. A citation that points across two pages is a citation nobody can check.
Local embeddings
all-mpnet-base-v2 at 768 dimensions, on the machine. ADR-003 records the reason: no per-document API cost. A hosted model is used for classification and composition only.
Duplicates are caught twice, for different reasons
The same document arriving twice is common. The same content arriving as a different file is more common still, and more expensive, because it silently doubles the weight of one source in every future answer.
Six decisions worth a technical reader’s attention
Refusal is a designed feature
Not an error branch. Below 0.35 the Response Composer is never called and the answer is an honest empty one. It is the cheapest path through the system, not the most expensive.
Filters run inside the query
The status = COMPLETED payload filter is part of the Qdrant search, not a list comprehension afterwards. That is the difference between correct retrieval and retrieval that looks correct on a small corpus.
No agent failure reaches the user
Three agents, each with an explicit timeout and a documented fallback. Identity extraction: 30s, returns null. Query Planner: 25s, zero retries, falls back to searching the raw question. Response Composer: 60s, one retry, falls back to a template over the retrieved passages.
Citations are verified, not trusted
Cited IDs are intersected with the IDs actually supplied to the model, then intersected again against live documents rows. A model that invents a source cannot produce a working citation.
Point IDs are deterministic
uuid5(namespace, "documentId:chunkIndex"). Reprocessing a document overwrites its own points rather than duplicating them, and the collection is created once and never recreated (ADR-006).
The test suite enumerates its own routes
One test walks the application route table and asserts every endpoint returns 401 without a token. A new endpoint cannot be added without authentication and quietly pass review.
Security is layered, and it fails closed
jti denylist fails closed, so a Redis outage revokes access rather than granting it.| Control | Implementation |
|---|---|
| Passwords | bcrypt at 12 rounds. Login enumeration defeated by identical body, status and timing. |
| Rate limits | Login 5/min per IP · upload 20/min per user · chat 30/min per user · 300/min default |
| Upload validation | Extension, MIME type, magic-byte signature and size. 20 MB documents, 2 MB images. |
| Headers | nosniff · X-Frame-Options: DENY · CSP · Referrer-Policy · HSTS in production |
| PII | PAN and Aadhaar redacted in every log sink, and masked in chat answers before storage or return. |
| Data | Ten tables, each indexed to its query pattern, with an append-only audit log carrying user, request ID, IP and user agent. |
What it costs to run
Most RAG cost estimates ignore ingestion and assume every question reaches the model. This one is counted per operation, which is why the refusal path matters commercially and not only editorially.
| Operation | Model calls |
|---|---|
| Ingesting a standard document | 3 model calls |
| Ingesting an identity document | 4 model calls |
| Answering a grounded question | 2 model calls |
| Refusing an ungrounded question | 1 model call |
| Embedding, at any volume | 0, runs locally |
Fourteen known limitations, documented in the open
The repository lists what the system does not do. Among them: chunk text is stored in
plaintext, there is no virus scanning on upload, no redaction before text reaches the model provider, prompt
injection is reduced rather than solved, the encoder truncates silently past 384 word-pieces,
all-mpnet-base-v2 is English-centric, there is no reranking, fixed-window rate
limiting allows double the intended rate across a boundary, Redis failures fail open for context
and rate limits, there is no answer streaming, no Playwright end-to-end suite, and no coverage
gating.
We publish that list deliberately. Every system has one. Most are discovered by the client six months in. You should expect the same candour about anything we build for you.
Apply this discipline to your retrieval
Sixty minutes on your data, your queries and where a threshold belongs in your system.