Found But Not Read: Why Vision-Language Models Still Benefit From OCR

The optical illusion of visual document search

Picture this scenario. You build a document question-answering pipeline around a visual retriever like ColQwen2. You render PDF pages as images and retrieve pages based on the user's question. Your retriever pulls the top sixteen page images and hands them directly to a vision-language model—a VLM, like Qwen2.5-VL or Qwen3-VL—to read and answer.

The architecture sounds clean. By feeding page images straight to a multimodal model, you preserve the document's visual presentation without requiring an OCR transcript as the reader's input. The underlying assumption feels self-evident: finding the right page is the hard part; once the page image sits inside the context window, reading the answer should be straightforward.

A research paper from Harbin Institute of Technology, titled Found but Not Read: When Extracted Text Closes the Retrieval-Reading Gap in Document Vision-Language Models (Xia et al., October 2, 2026), puts that assumption to the test.

The authors show that even when a visual retriever achieves near-perfect recall, placing the pages holding the answer directly in front of the model, the tested VLMs still leave substantial evidence unused. They call this failure the retrieval-reading gap: evidence found by the search system but not successfully used by the reader.

The surprising part is the intervention. Take those same page images, run CPU OCR over them, and add selected text blocks to the prompt alongside the images. On FoveDoc-Bench, strict answer accuracy rises by 13 to 16 percentage points with the main reader. The visual model already had the pixels. Presenting their textual content separately helped it use what was there.

Trapping the reader: The paired protocol on FoveDoc-Bench

In document benchmarks, diagnosing a failed question can be difficult. If your pipeline returns the wrong answer, did your search engine retrieve the wrong pages, or did your language model fail to use the right page?

To isolate the reader from the retriever, Xia and his co-authors created a benchmark called FoveDoc-Bench. It consists of 2,346 questions from 1,173 born-digital documents, each supplying a 16- to 24-page context. Every question links to annotated source blocks and exact text excerpts. The documents are held out from the study's auxiliary training and validation QA pools.

They split the benchmark into two distinct tasks:

1. V-NIAH: The answer depends on one evidence block on one page.
2. V-MQAR: The answer requires joining two evidence blocks from distinct pages.

Next, they established a retrieval-saturated protocol. Using ColQwen2 to retrieve the top 16 pages for each question, full recall@16—the fraction of questions for which every gold evidence page was retrieved—hit 1.000 (100%) on V-NIAH and 0.998 (99.8%) on V-MQAR. Across 2,346 questions, the model had every gold evidence page in its prompt for 2,344 of them. Retrieval failure was almost entirely removed from the comparison.

With retrieval held constant, they tested the same frozen model, stock Qwen3-VL-2B-Instruct, under two conditions:

• Arm A (Images alone): The model received the question and the 16 retrieved page images.
• Arm B (Images + Text Memory): The model received the question, the same 16 page images, and the 16 text blocks from those pages that scored highest lexically against the question. Blocks were grouped by page, tagged with page numbers, and capped at 300 characters each.

Because Arm B removes nothing and adds only text extracted from pages already present, the accuracy difference measures the benefit of making that content separately accessible.

Answers were graded by an LLM judge, with strict accuracy counting only verdicts of correct. The main and external comparisons used GPT-5.4; localization and retrieval controls used GPT-5.6 Luna. The judges agreed on 98.8% of strict verdicts over 1,963 identical predictions. Two authors also checked 200 samples drawn from disagreements between the image and OCR arms, agreeing with the judge on 96%.

The score: A 13 to 16 point leap from CPU OCR

The baseline performance of the image-only pipeline was surprisingly fragile.

On the single-needle task, V-NIAH, Qwen3-VL-2B answered correctly 35.0% of the time: 410 out of 1,173 questions, despite receiving the evidence page. Adding text extracted by RapidOCR, using PP-OCR models on CPU, pushed accuracy to 50.6%, or 594 out of 1,173. That is a +15.7 percentage point gain (p = 2.8e-24).

The relative jump was even larger on the two-page task, V-MQAR. The visual reader alone scored 15.3%, or 180 out of 1,173. Adding CPU-extracted text nearly doubled its performance, lifting accuracy to 28.7%, or 337 out of 1,173: a +13.4 percentage point gain (p = 3.3e-26).

When the authors replaced OCR text with the native PDF text layer—the exact born-digital reference for these pages—accuracy climbed to 69.1% on V-NIAH (+34.2 points over images alone) and 43.2% on V-MQAR (+27.9 points).

Consider a concrete example from the paper. On page 12 of a 23-page Cloudflare blog-post context, a code comment states that the TLS model uses M4 preprocessor macros for notation.

The question asks: "What specific type of preprocessor macros does the TLS model use for notation?"

Given the page images alone, Qwen3-VL-2B answered "Mk". When the prompt included the RapidOCR block (score 22.7, p12_b5) containing uses M4 preprocessor macros, the model returned "M4".

Similarly, on a two-page question about Linux kernel tracing commands on pages 26 and 32, the image-only model returned tcp_sendmsg and tcp_write_timer. Adding OCR text produced the correct pair: tcp_sendmsg and mod_timer.

Why did CPU OCR retain roughly half the gain of the PDF text layer—46% on V-NIAH and 48% on V-MQAR?

The authors checked whether OCR was simply supplying less text. It was not. The born-digital layer fragmented into about 53 blocks per page with a median length of six characters, whereas OCR grouping produced about 17 blocks per page with a median length of 23 characters. At the fixed block budget, OCR memory contained more characters on 77.1% of samples, with an average difference of +894 characters.

The limitation was evidence fidelity and completeness. Measured against the gold excerpts, the best single packable OCR block recovered 76% of an excerpt's tokens on average. It recovered at least 90% for 53% of evidence items on both tasks. The authors connect OCR's smaller accuracy gain to its incomplete recovery of the evidence, despite its larger text volume.

Scale and prompt tricks did not close the gap

A natural reaction is to suspect that this is mainly a symptom of using a compact 2-billion-parameter model. Would a larger reader make the extra text unnecessary?

The researchers tested six off-the-shelf vision-language models spanning three families:

• Qwen3-VL (2B, 4B, and 8B)
• Qwen2.5-VL (7B)
• LLaVA-OneVision (7B)
• Idefics3 (8B)

This comparison used PDF text-layer memory on the first 400 samples per task, rather than CPU OCR on the full benchmark. Every model showed statistically significant gains across both tasks (p < 1e-13 in all twelve model-task combinations).

On LLaVA-OneVision-7B, adding text improved accuracy by 29.2 points on V-MQAR and 38.5 points on V-NIAH. On Idefics3-8B, the gains reached 42.0 and 61.3 points. Those numbers need a qualification: image splitting was disabled or pixels capped for these readers to fit 16 pages. The authors therefore compare the direction of the benefit across families, rather than treating the gain sizes as directly comparable.

Within Qwen3-VL, where the three sizes were evaluated at identical settings, scaling helped but did not eliminate the gap:

• 2B reader: +28.2 pp gain on V-MQAR / +36.0 pp on V-NIAH
• 4B reader: +25.0 pp gain on V-MQAR / +30.7 pp on V-NIAH
• 8B reader: +23.0 pp gain on V-MQAR / +22.5 pp on V-NIAH

A larger reader benefited less from the added text, but at 8 billion parameters the advantage still exceeded 22 percentage points on both tasks.

The researchers also tested several changes to selection, prompting, and fine-tuning:

• With a 4B reader on V-MQAR, doubling the block budget, using a union selector, and switching to a reasoning-mode reader each moved accuracy by at most +3 percentage points.
• Self-ask decomposition reduced accuracy by 4.3 percentage points in that setting.
• LoRA fine-tuning recovered 6 points on V-MQAR and 5 points on V-NIAH for the 2B model; the reported 4B LoRA gain was 0.9 points.

These tested interventions were small relative to the text-layer gap. They do not establish that scaling or tuning can never solve it, but they show that the particular alternatives tried here did not.

The diagnostic breakthrough: The model can read, but struggles to find

To understand why readers fail on pages they already hold, the authors ran three diagnostic comparisons.

First, they supplied the full OCR transcript of all 16 pages in reading order, without selecting blocks based on the question. Accuracy improved by 17.9 points on V-MQAR and 22.1 points on V-NIAH. That exceeded the gains from the 16-block digest under a neutral prompt: 14.1 and 16.5 points, respectively. Question-conditioned selection was therefore not necessary for a substantial benefit.

Second, they supplied those selected blocks as image crops, appended to the full pages without extracted text. The crops used the reader's own page pixel density, so the intervention isolated the blocks without increasing their pixel density.

The crops recovered 87% of the neutral-prompt text digest's gain on V-MQAR (+12.3 points) and 92% on V-NIAH (+15.2 points).

Third, they drew red boxes around the same blocks on the full-page images. The boxes produced no comparable recovery: +0.5 points on V-MQAR and a decline of 3.0 points on V-NIAH.

Together, the comparisons look like this:

Intervention added to the 16-page input    V-MQAR gain    V-NIAH gain
Red boxes around selected blocks            +0.5 pp        -3.0 pp
Selected blocks as image crops             +12.3 pp       +15.2 pp
OCR digest under a neutral prompt          +14.1 pp       +16.5 pp
Full OCR text in reading order             +17.9 pp       +22.1 pp

These results support an evidence-accessibility bottleneck. The reader can use much of the information when it is lifted out of the surrounding page, even when the information remains pixels.

The experiments do not identify a specific attention component as the cause. Nor do they prove why red boxes fail. What they establish is the behavioral difference: marking regions on a full page did little, while presenting those regions separately recovered most of the digest's benefit.

Page-level retrieval is not evidence-level retrieval. Retrieving a page brings the haystack into view; presenting selected text or crops can make the needle easier to use.

Guardrail one: The flowchart trap and evidence modality

At this stage, you might be tempted to conclude that every document question should receive an OCR transcript. The paper identifies two boundaries that make that conclusion too broad.

The first involves evidence modality: whether the answer depends on text or on graphical structures such as charts and figures.

The authors evaluated the paired protocol on MMLongBench-Doc, which contains 1,091 questions across 135 documents, reaching up to 468 pages. Unlike FoveDoc-Bench, its evidence is mostly graphical, and retrieval is not saturated: full recall@16 is 84.4%.

Among the 838 answerable questions with annotated evidence, adding OCR memory produced a negligible aggregate change of +0.4 percentage points (+0.004, p = 0.86). That result applies to the answerable, annotated stratum, rather than all 1,091 questions, which also include deliberately unanswerable items.

When the authors separated the 414 questions carrying exactly one evidence tag, a different pattern emerged:

• Text-only evidence (129 questions): Adding extracted text yielded a +8.5 percentage point gain.
• Chart or figure evidence (285 questions): Adding extracted text yielded a −4.6 percentage point change.

The difference between these effects was 13.1 percentage points (p = 0.0024), the external modality contrast that survived the paper's correction for multiple comparisons.

The overlapping evidence categories showed a similar ordering:

• Plain text: +5.2 pp
• Tables: +1.9 pp
• Layout: +0.9 pp
• Figures: −2.8 pp
• Charts: −4.0 pp

These category estimates describe the observed pattern; they are not all independently established benefits or harms.

Consider sample #44 from MMLongBench-Doc. The question asks: "According to the flowchart 'Analytics Value Chain', what comes between Reporting and Action?"

The image-only model answered correctly: "Analysis".

The packed OCR memory from page 13 omitted Analysis and the arrows, retaining fragments including Reporting, Value, and Action. With that memory added, the model answered "Action".

The example shows how a transcript can omit the relationships and labels needed to interpret a graphic. It does not reveal the model's internal reasoning, but the outcome is clear: the added text accompanied a correct-to-incorrect answer change.

Flattened text can lose the spatial structure that makes a diagram meaningful. In this evaluation, text evidence benefited, while chart and figure evidence showed a negative average effect.

Guardrail two: Text amplifies retrieval, but does not replace it

The second boundary concerns retrieval quality. If your retriever misses the evidence page, how much can text from the remaining pages help?

To find out, the researchers degraded retrieval under controlled conditions. They removed gold pages from the top 16 results and substituted pages that ColQwen2 had ranked below the top 16, keeping the page budget fixed.

The eligible subset contained 938 V-MQAR questions and 1,025 V-NIAH questions whose documents could supply substitutes at every tested level.

Retrieved gold pages	V-MQAR gain	V-NIAH gain
100% recall	+12.5 pp	+14.9 pp
50% recall	+6.6 pp	—
0% recall	+3.0 pp	+4.3 pp

As recall dropped to zero, the text advantage fell by 9.5 points on V-MQAR and 10.6 points on V-NIAH. Both within-sample changes were statistically significant (p < 1e-9). Roughly three quarters of the full-recall advantage depended on having the gold pages present.

Part of the residual at zero recall involved answer-string leakage. The gold answer string still appeared in the packed memory from non-gold pages for 4.8% of V-MQAR samples and 12.4% of V-NIAH samples.

After filtering those cases using normalized answer-string matching, a 2.8-point advantage remained on both tasks. The residual stayed between 2.5 and 3.1 points as the detector's minimum answer length varied from four to twelve characters. Leakage therefore explained only part of the remaining benefit.

To test whether the text format itself helped, the authors replaced each document's memory with text from another document under the same packing rule. No document received its own text, and paired documents came from different source families.

At zero recall, foreign text produced no detectable benefit over images alone: +0.0 points on V-MQAR and −0.9 points on V-NIAH. At full recall, its effects were also statistically indistinguishable from zero: +1.9 and −1.7 points.

These controls support the authors' interpretation that the remaining benefit comes from the document's own non-gold context. Merely adding text of the same form was insufficient.

Extracted text helps principally by making retrieved evidence accessible, with a smaller contribution from related document context. It is an amplifier of retrieval that works, rather than a substitute for finding the evidence.

Why it matters: Architectural lessons for production document AI

For teams designing document search and question-answering systems, the paper offers useful design guidance. Its measured gains belong to these tasks, readers, and input settings; applying the lessons elsewhere requires evaluation on the documents and questions that matter to your system.

1. Keep extracted text in contention for dense textual evidence

The study gives a concrete reason to test image-plus-text inputs even when a VLM can accept pages directly. On FoveDoc-Bench, CPU OCR added 13 to 16 points with Qwen3-VL-2B, and PDF text-layer memory helped all six tested readers.

Those are not guaranteed gains for contracts, financial filings, or other production collections. They show that successful page retrieval and visual input capability alone do not establish that a reader will use the evidence reliably.

2. Evaluate OCR over the retrieved pages

The tested pattern offers a practical starting point:

• Retrieve candidate page images with a visual retriever.
• Extract text from those pages; the paper used RapidOCR with PP-OCR models on CPU.
• Select and pack text blocks, then supply them alongside the original images.

The experiment used 16 retrieved pages and a 16-block memory budget. CPU OCR retained roughly half the gain of the native PDF text layer in the main comparison.

The paper does not establish query-time latency, total indexing cost, or production cost savings. Whether to extract on demand, cache results, or preprocess pages remains an implementation decision to measure.

3. Make evidence modality part of the decision

The chart and figure results argue against assuming that every extracted transcript will help. For graphical evidence, the representation may need to preserve structure that flat text omits.

A modality-aware pipeline is therefore worth testing: use extracted text where the answer depends on textual evidence, and assess visual or structure-preserving representations where relationships in graphics matter.

The study does not validate a particular layout detector or routing system. Its contribution is the boundary such a system should respect, rather than a universal rule to exclude all chart text.

4. Beware of aggregate benchmarks

On MMLongBench-Doc's answerable, annotated stratum, the aggregate gain was just +0.4 percentage points. Yet the single-tag comparison showed an 8.5-point gain on text and a 4.6-point decline on charts and figures.

Report results by evidence modality and retrieval recall. Otherwise, a change in the mixture of questions can conceal benefits, hide harms, or make one benchmark's aggregate a poor guide to another workload.

5. Measure retrieval and reading separately

Roughly three quarters of the OCR advantage in the controlled recall experiment depended on the gold pages being present. That makes retrieval quality central to deciding where extracted text will help.

Measure full recall@k independently, then compare image-only and image-plus-text answers on the same retrieved pages. The question is not just whether the system found the right page. It is whether the reader successfully used the evidence once it arrived.