Stop Sending Every Page to a Vision Model: Lessons from Citigroup, EY, and NVIDIA Researchers

When engineering teams build document pipelines for Retrieval-Augmented Generation (RAG), the conversation almost immediately turns into a model shootout. Should you call Gemini 3.1 Pro? Route pages to Mistral OCR v3? Use a compact model like Granite-Docling?

A paper by Abbas Raza Ali (Citigroup), Muhammad Ajmal Siddiqui (Ernst & Young), and Moona Zahid (NVIDIA), titled Smart Content Ingestion for Generative AI Workloads, brings a cold dose of production reality to this discussion. Published as a preprint on October 5, 2026, it is under review for MLSys 2027.

Reporting measurements on an internal corpus of 180 documents spanning 10,507 pages, the authors highlight two numbers that should inform how you design your ingestion architecture:

1. About 77% of extraction wall-clock time went to shared preprocessing in the fastest configuration. Image enhancement and layout parsing consumed roughly 2,062 seconds, compared with 611 seconds of Mistral OCR inference. In the slowest configuration, Granite-Docling inference took roughly 1,350 seconds, leaving preprocessing at about 60% of total extraction time.
2. About 92% of pages took the native extraction route. Out of 10,507 pages, 853 crossed the system's image-area threshold for page-level OCR. The remaining 9,654 pages followed the native path at near-zero model cost, although extracted figures could still trigger vision captioning.

If you pipe every document wholesale into a vision API, this architecture suggests an alternative: extract directly accessible content first, and reserve page-level OCR for image-dominant pages. Document ingestion is not just a prompt engineering exercise. The paper treats it as a four-stage systems pipeline where upstream parsing errors can leave downstream retrieval without the information it needs.

The perception layer of agentic systems

In traditional machine learning, data preparation was narrow and schema-bound. Tasks, data representations, labels, and model architectures were closely coupled.

Generative AI changed that relationship. Foundation models support multiple downstream tasks through natural language instructions and context. As the model side became more general, the data side brought its own challenge: enterprise knowledge lives in PDFs, slide decks, spreadsheets, scanned documents, forms, tables, and diagrams.

These files pack textual, visual, geometric, and structural information into a single artifact. If an ingestion parser flattens a two-column document into interleaved lines, or loses the relationships inside a financial table, downstream components receive a damaged representation. A vector retriever cannot reliably recover information absent from its index. A re-ranker cannot restore column alignments that extraction discarded.

The authors position measured content extraction as the perception layer of enterprise agentic systems. Their argument is that downstream reasoning depends on representing the source correctly at this boundary.

The paper presents a production architecture split into four sequential phases:

• Phase 1: Multimodal Content Extraction, combining selective OCR routing with heuristic native extraction.
• Phase 2: Curation and Extraction Evaluation, using multi-label scarcity-first sampling to audit parsers against ground truth.
• Phase 3: Structure-Aware Parent-Child Chunking, decoupling retrieval units from reading contexts.
• Phase 4: Read-Only Retrieval Evaluation, generating grounded synthetic questions to test retrievability.

Three engineering principles govern the design: structure before semantics, never mutate what you measure, and budget your labels.

Selective routing: Let PyMuPDF carry the heavy traffic

The core efficiency insight in Phase 1 is simple: extract directly accessible content without OCR, and use an image-area heuristic to route image-dominant pages to vision models.

For every incoming PDF page $p$, the system computes an image-area coverage ratio:

$$r(p) = \frac{A_{\text{img}}(p)}{A(p)}$$

If the total area of embedded images $A_{\text{img}}(p)$ exceeds 80% ($\tau_{\text{cov}} = 0.8$) of total page area $A(p)$, the page routes to OCR. Otherwise, it stays on the native extraction path powered by PyMuPDF.

On the authors' Orion dataset of 180 documents, 853 out of 10,507 pages (8.1%) took the OCR route. The remaining 91.9% followed native extraction. This is an observed routing split on Orion, not an estimate for enterprise documents generally.

Separately, the curation engine applies a two-signal test when labeling scanned pages:

$$s(p) = 1_{|T(p)| < \kappa ;\wedge; r(p) > \tau_{\text{cov}}}$$

With $\kappa = 40$, a page must have fewer than 40 extractable characters as well as high image coverage to receive the scanned label. This prevents a page with a large photograph and a substantial text layer from being mislabeled as scanned during curation. It does not change the extraction router's area-only rule.

For image-dominant pages, the paper describes an optional enhancement sequence: border removal, noise estimation via Laplacian variance to choose among Gaussian, bilateral, or non-local-means filters, Contrast Limited Adaptive Histogram Equalization (CLAHE), unsharp masking, and cubic upscaling. Rasterized pages then go to an OCR back-end. The inference infrastructure includes retry and back-off handling for 429/5xx responses and OAuth2 token refresh on 401.

Deterministic layout reconstruction without deep learning

For native-text pages, the structural reconstruction stage avoids learned models. Instead, it relies on deterministic heuristics:

• Font size calibration: The extractor calculates the modal font size $f^$ across the document, treating it as standard body text. Distinct font sizes satisfying $f \ge 1.12 f^$ become heading candidates. These sizes are sorted descending into hierarchical tiers, up to a configured depth, to emit Markdown headings.
• Header and footer suppression: The system inspects text blocks within page margins, normalizes digits so that "Page 4" and "Page 9" share a key, and drops matching margin content recurring on more than 50% of pages.
• Two-tier table detection: Ruled tables are captured using lines_strict in PyMuPDF. When the table strategy is set to auto, a word-alignment detector also looks for borderless tables by clustering words into rows and grouping columns using an x-alignment tolerance. A guard rejects side-by-side prose that resembles a table.
• Line-granularity reconciliation: Rather than discarding an entire text block that overlaps a detected table, this pipeline tests each line individually. A line is suppressed only if more than 50% of its area sits inside the table rectangle, preserving nearby sentences that merely share a text block with a table fragment.
• Raster de-duplication: Embedded raster images larger than $32 \times 32$ pixels are de-duplicated using MD5 hashes and represented by figure placeholders such as ## Figure-{page}.{n}. Remaining placeholders trigger vision captioning, with Gemma-3 and Llama-4-Scout among the models shown in the architecture.

The output of Phase 1 is tagged Markdown preserving explicit ### Page: N, table, and figure markers. Native structural reconstruction and vision captioning are distinct parts of that process.

The scarcity trap: Auditing extraction when 90% of data is text

Before piping extracted Markdown into an embedding model, you need to know whether your extractor actually worked. This brings us to Phase 2: data curation and extraction evaluation.

Evaluating document extraction across enterprise repositories encounters a stubborn sampling problem. If most pages are plain text, a random sample can leave scanned pages, diagrams, and infographics with too few examples to estimate extraction quality. Rare modalities need deliberate representation in the evaluation set.

The authors formalize this as a covering problem with multiple labels per page. Their taxonomy includes Scanned, Text, Table, Diagram, Graph/Infographic, and Photo/Signature categories.

Given target quotas $t_m = \lceil \phi_m N \rceil$ for each modality across a total budget of $N$ pages, Algorithm 1 applies a scarcity-first greedy heuristic:

1. Sort modalities in ascending order of their target quota. This order follows configured quotas, not necessarily their observed frequency in the corpus.
2. For the active modality, rank candidate pages by a helpfulness score: the number of other still-unmet quotas a page also satisfies.
3. Select pages while that modality has an unmet quota and candidates remain, updating deficits across all matching labels.
4. Top up with unselected pages or trim low-helpfulness selections to reach the page budget, then report target versus achieved coverage.

The method is a heuristic, not a guarantee that every quota can be met. Selected pages can be exported as isolated single-page PDFs or whole source documents when context matters. A six-sheet workbook records the configuration, selections, statistics, and achieved distribution. Ground-truth digitization and human validation are then performed on the sample.

Scoring text and table fidelity

The evaluation harness normalizes text through Unicode NFKC, case-folding, punctuation and ligature unification, Markdown and HTML stripping, and whitespace collapsing. It computes Character Error Rate (CER) and Word Error Rate (WER) via Levenshtein distance, along with precision, recall, F1, bag-of-words F1, smoothed BLEU-4, and a length ratio $(|\hat{c}| / |c|)$.

That ratio flags possible over-production or content loss. It does not, by itself, establish whether extra text is duplication or hallucination.

For tables, the scorer adds structural comparison using Tree Edit Distance-based Similarity (TEDS). Pipe and HTML tables are parsed into rectangular grids and modeled as trees (table -> row -> cell). The reported similarity takes the form:

$$\text{TEDS}(T_a, T_b) = 1 - \frac{\text{EditDist}(T_a, T_b)}{\max(|T_a|, |T_b|)}$$

This measures table organization and, in the content-sensitive mode described, differences between cell strings.

Finally, back-ends are ranked using a composite score ($Q_m \in [0, 100]$):

$$Q_m = 100 \cdot \frac{\sum_j w_j \mu_{j,m}}{\sum_j w_j} \cdot \text{cov}_m$$

The metrics are oriented toward accuracy and combined using configurable weights. The variable $\text{cov}_m$ represents the fraction of expected files for which the model produced output. Missing files reduce the score proportionally, explicitly penalizing an engine that skips difficult documents.

The result is a configurable evaluation score, not a standalone percentage of correctly extracted content.

Parent-child chunking: Small cards inside large folders

Phase 3 addresses chunking. The paper describes two competing failure modes:

• Large chunks can dilute a relevant passage's contribution to an embedding. Under the paper's mean-pooling approximation, one relevant sentence contributes less as the number of sentences grows.
• Small chunks can fragment an answer across pieces that are individually poor retrieval matches.

The architecture addresses this trade-off by decoupling the retrieval unit from the context unit.

Retrieved units are children: focused index cards targeting 320 tokens, with a normal prose ceiling of 512 tokens. Context units are parents: structural folders with a configured maximum of 3,500 tokens. These are not unconditional limits: overlong individual sentences and atomic tables or figures are preserved whole, and an oversized table or figure can also exceed the parent limit.

Children are embedded with GTE, L2-normalized, and indexed in FAISS using an exact inner-product index (IndexFlatIP). Parents are stored in a key-value map from parent_id to parent text.

At query time, vector search retrieves child cards. Expanding a child to its stored parent requires one $O(1)$ lookup. That complexity describes the context lookup, not the vector search or the complete query.

Document Markdown
       │
       ▼
[Parent Section Folder: configured maximum 3,500 tokens]
       │                                      (Key-Value Store)
       │
       ├──► Child Card 1: target ~320 tokens (FAISS Index)
       │      Optional prefix: "Doc > Section > Subsection (p.4)"
       │
       ├──► Child Card 2: target ~320 tokens (FAISS Index)
       │      Optional prefix: "Doc > Section > Subsection (p.4)"
       │
       └──► Child Card 3: whole atomic table (FAISS Index)

The four packing rules

Sentence packing inside pack_sentences() follows four rules:

• Rule 1 (Token budgeting): Target roughly 320 tokens using the GPT-2 tokenizer, with a 512-token ceiling for ordinary prose packing.
• Rule 2 (Sentence preservation): Never cut mid-sentence. Split on punctuation while preserving trailing quotes and brackets. If one sentence exceeds the ceiling, emit it as an isolated chunk rather than truncating it.
• Rule 3 (No crumbs): If a packed group falls below the minimum token threshold, merge it into the preceding chunk's non-overlap sentences when the combined size respects the ceiling.
• Rule 4 (Atomic tables and figures): When the packer encounters a table or figure, it flushes pending prose and emits the entire structural unit as one child, including when that unit exceeds the normal token budget.

When a chunk closes because the next sentence would exceed the ceiling, the system carries up to 48 tokens of trailing whole sentences into the next chunk.

To add positional context without an LLM rewrite, optional breadcrumbs prefix each child's embedding text with its structural address: Document > Title > Section > Subsection (p.N). The raw, unprefixed text is preserved separately for display.

Auditing retrieval: 25,050 questions with a read-only evaluator

Phase 4 closes the loop with RetrievalEvaluator. Following the principle of never mutate what you measure, the harness opens the index using faiss.IO_FLAG_READ_ONLY. It reuses the exact GTE embedding fingerprint and page-tag grammar used during indexing.

The evaluator generates grounded synthetic questions using Mistral-Medium for pages containing at least a configured minimum number of tokens. Its prompt requests specific fact-level questions and broad questions synthesizing content across the page, requires raw JSON, and forbids outside knowledge. These instructions constrain generation; they do not independently prove every generated question is valid.

Each question is embedded using GTE and searched within its source document using FAISS IDSelectorBatch, with an exhaustive-filter fallback when selector support is unavailable. Retrieved children are de-duplicated by parent_id, keeping the highest-scoring child per parent. The search over-fetches candidates to support returning distinct parents.

A hit is registered if the candidate child clears a cosine similarity threshold $s(q, c) \ge \theta$ and its source page falls within one page ($\delta = 1$) of the question's origin:

$$\text{hit}(q, c) = 1_{s(q, c) \ge \theta ;\wedge; |\pi(c) - \pi(q)| \le \delta}$$

On the Orion corpus, the examiner generated and evaluated 25,050 questions across 180 documents:

• hit@1: 68.58%
• hit@3: 82.30%
• hit@5: 87.66%
• hit@10: 92.77%
• Mean Reciprocal Rank (MRR): 0.7745
• Median Rank: 1.0 (Mean Rank: 1.81)
• Questions missed at all evaluated ranks: 1,810 (7.2%)
• Mean Per-Query Latency: 0.81 seconds, reported as dominated by remote embedding calls

Over two-thirds of questions met the evaluator's hit criterion at the first returned parent. By ten distinct parents, that share reached 92.77%.

Those are retrieval-proxy results. The criterion checks similarity and page proximity, not an independently labeled exact target section or the correctness of a generated answer. Because search is restricted to the source document, the figures also do not measure finding the right document across the full corpus.

The extraction benchmark: Larger back-ends vs. compact models

The authors report extraction scores for four production back-ends. Their measurements come from the system's own evaluators on internal corpora; they are not an independent public leaderboard.

Model	Composite Score ($Q_m$)	Character Error Rate (CER)	Word Error Rate (WER)	Table Similarity (TEDS)	Inference Time on Orion	Total Wall-Clock Time on Orion
Gemini 3.1 Pro	97.37	0.13%	0.13%	0.9952	Not separately reported	Not separately reported
Mistral OCR v3	95.98	1.39%	1.55%	0.9737	~611 s	2,672.67 s
Granite-Docling (258M)	92.92	2.60%	2.90%	0.9599	~1,350 s	3,411.86 s
SmolDocling (256M)	90.70	4.43%	4.82%	0.9274	Not separately reported	Not separately reported

Note: Extraction wall-clock totals include approximately 2,062 seconds of shared preprocessing, described as image enhancement and layout parsing. The paper places the other two back-ends between the reported timing extremes without giving their individual times.

Gemini 3.1 Pro led the reported accuracy ranking, recording a CER of 0.13% and a TEDS score of 0.9952.

Mistral OCR v3 finished second with a composite score of 95.98. Its CER of 1.39% was roughly an order of magnitude higher than Gemini's, while it delivered the fastest reported extraction time. Its model inference component took about 611 seconds, compared with about 1,350 seconds for Granite-Docling.

The compact models, Granite-Docling and SmolDocling, trailed Gemini by approximately 4.5 and 6.7 composite points. Their reported CERs were 2.60% and 4.43%, respectively; their TEDS scores were 0.9599 and 0.9274. These are aggregate benchmark results, not separate measurements of plain body text or particular table difficulty levels.

The paper also cites earlier CNN table detection and Encoder-Dual-Decoder cell extraction work reporting 85% word accuracy on image tables. That provides historical context, but the supplied results do not establish a matched-dataset comparison from which to calculate a controlled improvement over that earlier system.

Why it matters: The real trade-offs for production document pipelines

If you are an engineering lead or product manager building document ingestion for enterprise search or agentic workflows, this paper offers several operational lessons:

1. The cheapest OCR call is the one you never make

Selective routing kept 91.9% of Orion pages off the page-level OCR path. That is a concrete result worth testing against your own documents.

It is not a measured 91.9% reduction in total cost. Native extraction still has processing costs, and figure captioning can still invoke vision models. The paper does not provide per-page prices or a before-and-after billing comparison. The supported lesson is narrower: directly accessible text need not automatically receive page-level OCR.

2. Shared preprocessing deserves a place in your latency budget

Figure 4 makes the extraction timing visible: roughly 2,062 seconds went to shared image enhancement and layout parsing, compared with 611 seconds of Mistral OCR inference.

That puts preprocessing at about 77% of extraction wall-clock time in the fastest configuration and about 60% in the slowest. The authors identify it as the next optimization target.

The measurements do not establish CPU utilization, serial execution, or the speedup a particular multiprocessing or image-filter change would deliver. They do show why choosing the fastest OCR back-end leaves a substantial part of extraction time untouched.

3. Character error rate is only part of extraction quality

A text score does not provide a complete account of table fidelity. The paper evaluates both, combining character- and word-level measures with table structure and cell-related metrics.

When comparing extractors, preserve that distinction. TEDS supplies structural evidence alongside CER and WER; the composite score combines those signals but should not conceal them.

4. Separate your retrieval cards from your context windows

The paper's parent-child architecture offers a concrete way to match small passages while returning broader context. Structural tags keep tables and figures intact, and optional breadcrumbs add document position to the text being embedded.

The Orion run demonstrates that this design can be evaluated at scale. It does not include a controlled retrieval comparison proving that parent-child chunking outperforms fixed-size, semantic, or other alternatives on the same corpus.

Where the system hits its limits

The authors describe the system as production-ready, but the evidence supports several boundaries on how to read its results:

• Synthetic question dependence: The retrieval measurements use questions generated by Mistral-Medium. The paper explicitly notes that this proxy depends on the generator's quality. It does not report validation against a set of real user queries.
• Document-scoped, page-based hits: Search takes place within the question's source document, and success depends on similarity and page proximity. These results do not establish corpus-wide document selection or answer correctness.
• Internal evaluation and limited comparisons: The reported scores are self-measurements on internal corpora. The paper does not report a chunking ablation or a controlled retrieval comparison against the alternative strategies it discusses.
• Configurable routing heuristics: The 80% image-area threshold produces the reported routing split on Orion. The paper does not establish its routing accuracy across other document populations, and the stricter scanned-page labeling rule belongs to curation rather than extraction routing.
• Monitoring remains partly a proposal: Reference-based CER and WER require ground truth. The paper discusses reference-free signals and calibration strategies, but labels the monitoring rationale as a hypothesis or future direction rather than shipped calibration code. Such signals cannot report a true error rate on their own.

The takeaway

Document ingestion deserves more attention than a utility script assembled once and forgotten. In the authors' framing, it is the perception layer of a generative AI system.

By combining selective routing, deterministic layout reconstruction, structure-aware parent-child chunking, and read-only evaluators, the paper presents a concrete way to make ingestion decisions explicit and measurable. Its strongest contribution is the connection between those components and the evidence they produce about themselves.

Before you spend another week tweaking prompts or testing re-ranking models, take a hard look at your ingestion layer. If your parser loses the information on the page, downstream retrieval starts with a problem it may have no way to repair. ===BODY===