When a company deploys a question-answering system over its document library—legal contracts, technical manuals, procurement policies, internal reports—the default assumption is that vector embeddings will do the heavy lifting. Encode every chunk, store it in a vector database, retrieve the most similar ones at query time, pass them to the LLM.
That assumption holds when document sets are small and queries are conversational. It breaks down at enterprise scale.
The problem isn't that embeddings are bad. It's that they're being asked to do work that cheaper, faster tools should handle first. When your document library contains thousands of PDFs, using vector similarity as your primary filter means every query travels through the most expensive step before you've ruled out irrelevant sources. Latency grows. API costs compound. And the retrieved chunks are often pulled from documents that have nothing to do with the question.
A more principled architecture—detailed recently in Towards Data Science—flips the sequence: structured filtering first, embeddings last, and a single LLM call at the end to consolidate.
The core idea is to treat document retrieval as a progressive narrowing problem, not a single-shot similarity search.
The first filter is the cheapest: exact and near-exact keyword matching. Before touching any embedding model, the system scans document metadata and content for terms that match the query. This is fast, deterministic, and works well for enterprise documents—which tend to use consistent vocabulary: contract clause names, product codes, regulatory article numbers, department labels.
A procurement team asking "what's the penalty clause for late delivery in supplier contracts?" doesn't need semantic search to rule out IT security policies and HR handbooks. A keyword pass eliminates irrelevant documents in milliseconds.
The second filter operates on document structure. Most enterprise documents—contracts, compliance manuals, technical specifications—have explicit hierarchy: section headings, numbered clauses, chapter markers. These are anchors: fixed points that signal what each section covers.
An anchor detector identifies these structural signals and maps queries to relevant sections before full-text retrieval happens. If the query is about "termination conditions," the system identifies which documents have sections explicitly labeled with that concept and prioritizes chunks from those sections. This is faster than embedding comparison and more interpretable—you can audit why a section was selected.
Running multiple anchor detectors in parallel—one for heading patterns, one for numbered clauses, one for bold definitions, one for cross-references—lets the system cover different document formats simultaneously without blocking on any single detection method.
Only after the keyword and structural passes have narrowed the candidate pool does the system apply embedding similarity. At this point, instead of searching across thousands of chunks, it's searching across a few dozen. The embedding model runs faster, results are more relevant, and the context window passed to the LLM is tighter.
The output of all three stages—filtered, ranked chunks with structural provenance—goes into a single LLM call. The model isn't asked to rank, select, and answer simultaneously. It receives a curated, structured context and produces a final response. This reduces hallucination risk and keeps token consumption predictable.
This isn't an academic architecture debate. For any mid-sized company in Mexico, Colombia, or Argentina operating with document-heavy workflows—legal, compliance, procurement, customer service, finance—the difference between a naive RAG implementation and a structured one shows up in three concrete places:
Cost. Running embedding inference and LLM completions on unfiltered document chunks at every query quickly becomes expensive. A structured pipeline meaningfully reduces LLM token consumption by shrinking the context window before the costly step—each query processes fewer chunks, and that savings compounds across hundreds of daily queries. For operations teams running document search at volume, this is a real budget line.
Accuracy. Enterprise queries are often precise: "What does clause 14.3 of the distribution agreement with Grupo Norte say?" A semantic search that retrieves loosely related chunks from different contracts produces a hallucinated synthesis. A structured search that anchors to document hierarchy, then applies embeddings, retrieves the right chunk from the right document.
Auditability. In regulated industries—financial services, healthcare, legal—knowing why an AI system retrieved a specific passage matters. Anchor-based retrieval provides a traceable path: this section was selected because the structural parser matched the heading pattern to the query intent. That's an audit trail, not a black box.
Most companies encountering poor RAG performance respond by switching vector databases, upgrading embedding models, or increasing chunk overlap. These are reasonable experiments, but they optimize the wrong stage of the pipeline.
The leverage is in what you retrieve before you embed, not in how well you embed. Document structure is information. Section headings, clause numbers, and table of contents entries were written specifically to describe content—they're dense signals that keyword and structural detectors can exploit cheaply.
Parallel detection matters because enterprise document libraries aren't uniform. A legal team's contract repository has different structural conventions than an engineering team's specification library. Running multiple structural detectors simultaneously, then aggregating their outputs before the final LLM call, handles heterogeneous collections without requiring one-size-fits-all formatting standards.
If your teams are manually tagging or annotating documents before ingesting them into a knowledge base, that's a signal your retrieval architecture is doing too little work upstream.
Whether you're building internally or evaluating vendors, these questions reveal whether a document intelligence system is architecturally sound:
The logic behind this architecture applies well beyond RAG: filter with cheap signals before applying expensive ones. Keyword matching costs almost nothing. Structural parsing scales linearly with document size. Embedding inference and LLM completion cost money and time.
A system designed around this sequence is faster, cheaper, and more traceable than one built on the assumption that vector search is sufficient on its own.
For the companies across LATAM ingesting years of accumulated documents into AI systems right now—contracts, manuals, and reports moving into knowledge bases to support agents and analysts—getting the retrieval architecture right before scaling is the decision that compounds. The foundation matters more than the model sitting on top of it.
At Xenturia, this is the kind of architectural decision we work through before recommending any document intelligence deployment. Getting the retrieval order right isn't a technical nicety—it's what determines whether the system performs at scale or fails quietly.
Schedule a free consultation with our team and discover how AI can transform your operations.
Schedule a consultation
Strategic AIAIWhat India's AI elephant detection systems teach LATAM leaders about real-time alerts, operational risk, and the architecture of prevention.
Strategic AIAIYour AI model can be corrupted without a breach alert. Here's how ML model poisoning works, where LATAM companies are exposed, and how to detect it.
Strategic AIAISignal's Meredith Whittaker says AI chatbots aren't your friends. For LATAM business leaders, that's the most useful design principle of 2026.
