RAG Architecture Services
Retrieval systems that stay accurate past the first ten thousand documents, with hybrid search, reranking and measured recall rather than hopeful embeddings.
Most RAG systems fail at retrieval, not generation
When a RAG system gives a wrong answer, the instinct is to blame the model or rewrite the prompt. In our experience the passage that contained the answer was usually never retrieved in the first place. No amount of prompt engineering fixes a retrieval miss.
So we build retrieval first and measure it on its own. Chunking strategy, hybrid keyword and vector search, reranking, and metadata filters are tuned against a labelled set of real questions before any answer-generation layer exists. Recall at k is a number we report, not a hope.
The failure modes change with scale, which is why the approach in our RAG architecture guide looks different from a weekend prototype. What works on five hundred documents falls apart at fifty thousand.
Teams that start here often pair it with vector search, custom AI chatbots and AI integration.
If the right passage is not in the top five, the model is being asked to answer from material that does not contain the answer. It will oblige, and it will be wrong.
Why RAG quality collapses as the corpus grows
Four issues that only appear once there is real content in the index.
Fixed-size splits cut sentences and tables in half, so the retrieved passage contains a fragment of the answer and none of its context.
Pure embedding similarity misses exact identifiers, product codes and error strings, which are precisely what technical users search for.
The top-k from the first pass goes straight to the model, so a marginally relevant passage outranks the correct one and nobody notices.
Documents change and the index does not, so the system confidently answers from a policy that was replaced last quarter.
What a RAG engagement delivers
A measured retrieval layer, a grounded answering layer, and the harness to keep both honest.
We look at what your documents actually are, since the right chunking for contracts is not the right chunking for changelogs or support tickets.
BM25 and vector search combined, so exact terms and semantic meaning both work, with metadata filters for tenancy and recency.
A cross-encoder pass over first-stage candidates, which in most engagements is the single biggest accuracy improvement available.
Labelled question and passage pairs, with recall at k and answer accuracy reported in CI so regressions are caught before release.
The answering layer may only use retrieved passages and must cite them, so every claim is checkable.
Change-data-capture pipelines that reindex what moved instead of rebuilding nightly, so answers stay current at scale.
From corpus audit to a measured retrieval system through controlled rollout
Retrieval is proven with numbers before generation is attached.
We inspect document types, structure, update frequency and access rules, and build a labelled question set with your subject experts.
Chunking, embedding and hybrid search implemented and tuned against the labelled set, reporting recall at k.
A cross-encoder pass added and measured, with the trade-off between latency and accuracy made explicitly rather than by default.
The answering layer is constrained to retrieved context and required to cite, then measured for faithfulness.
Incremental indexing, monitoring and a drift report so quality decay is visible before users complain.
The stack behind our retrieval systems
Postgres first, specialist vector databases only where the numbers justify them.
pgvector alongside your relational data keeps tenancy, permissions and joins simple at most realistic scales.
Hybrid first-stage search with a reranking pass, tuned per corpus rather than copied from a tutorial.
Model choice driven by measured recall on your corpus, not by benchmark leaderboards.
Retrieval and answer quality tracked separately, because they fail for different reasons.
What this looks like in production
Making forty years of manuals searchable
A manufacturer had four decades of maintenance manuals, service bulletins and part specifications, much of it scanned PDF. Field engineers were phoning head office for answers that existed in documents nobody could find.
The first prototype used fixed-size chunking and vector search alone. It scored low recall on a labelled set of real engineer questions, largely because part numbers and error codes are exactly what embeddings handle worst.
Structure-aware chunking, hybrid search and a reranking pass took recall to 89%. The generation layer barely changed. The accuracy came from retrieval, which is almost always where it comes from.
What a RAG architecture partner should measure
We report recall on your own labelled questions before we build the answering layer, because a RAG system that cannot find the answer cannot be prompted into knowing it.
Recall at k on your corpus is a number in every report, so quality is a metric rather than an impression.
A working demo and a written note every Friday covering what shipped, what slipped and what it means for the date. No status theatre.
Nothing goes live in one jump. We run in parallel or behind a flag until the numbers say it is safe to switch over.
