Donut Skips OCR Entirely, and It Still Reads Your Receipts

Picture a receipt scanner that never runs OCR. No text detection boxes, no character recognition step, no separate model bolted on to make sense of what the OCR engine spat out. You feed it a photo of a crumpled receipt and it outputs structured JSON: merchant name, date, line items, total. Sounds like it's skipping a step it needs. According to a 2021 paper out of NAVER (the company behind South Korea's dominant search engine), skipping that step is exactly what made the model faster and, in several benchmarks, more accurate than pipelines that use OCR properly.

The model is called Donut, short for Document Understanding Transformer. The name is a better acronym than most papers manage, and the idea underneath it is genuinely counterintuitive: the standard approach to reading documents runs OCR first and treats the recognized text as ground truth for everything downstream. Donut throws that assumption out and asks a transformer to go straight from pixels to structured output.

Sample from the CORD receipt dataset (Park et al., Clova AI / NAVER), licensed CC BY 4.0.

Why "just run OCR first" turns out to be the wrong plan

Every visual document understanding system, until recently, followed the same two-stage recipe. Step one: run an OCR engine (Tesseract, or a commercial API like Amazon Textract or Google Vision) to convert the image into text plus bounding boxes. Step two: feed that text into a second model that does the actual understanding, extracting fields, answering questions, or classifying the document type.

This works, and it's why OCR-based systems have been the default for years. But the paper's authors point to three problems baked into that design:

• Cost. Running a heavy OCR engine on every document, especially at scale, is expensive in both compute and latency. You're paying for a full recognition pass before your "real" model even starts.
• Inflexibility. Off-the-shelf OCR engines are tuned for specific languages and document types. Point one at a receipt in Korean or a densely formatted Japanese business card and accuracy can fall off a cliff. Retraining or fine-tuning an OCR engine for a new language is its own project.
• Error propagation. This is the one that actually bites in practice. If OCR misreads a "3" as an "8" on an invoice total, every downstream step inherits that error with no way to recover. The understanding model never even sees the original pixels. It just trusts whatever the OCR engine handed it, mistakes included.

That third point is worth sitting with, because it's the kind of failure that's invisible until it costs someone money. A two-stage pipeline can be individually well-tuned at each stage and still fail as a whole, because the handoff between stages throws away information (visual context, layout, font characteristics) that would have helped catch the error.

How Donut actually works: one model, straight from pixels to text

Donut's architecture is almost aggressively simple, which the authors treat as a feature, not an oversight. It's an encoder-decoder transformer, the same general shape used in many sequence-to-sequence models.

The encoder is a Swin Transformer, a vision model that processes the image in patches and builds up a visual representation. Crucially, there's no OCR module anywhere in this stage. It's not detecting text regions and then reading them. It's just building a general visual understanding of the whole image, the same way a vision model built for classifying photos of dogs would.

The decoder is a BART-style text transformer that takes that visual representation and generates output token by token, the same way a language model generates text one word at a time. The key trick is what it generates. Instead of just transcribing the text it sees, Donut is trained to output a structured sequence, essentially a flattened JSON-like format, that directly represents the answer to the task at hand. For a receipt, that means merchant name, item list, and total, formatted as a sequence of special tokens the model learned to produce during training. There's no intermediate "here's the text I detected" step. Pixels go in, structured fields come out.

Training a model to do this from scratch requires an enormous amount of labeled document data, which doesn't exist in the quantities needed. So the authors built a synthetic data generator called SynthDoG (Synthetic Document Generator) that produces realistic-looking documents with corresponding ground truth, in multiple languages, by combining background images, sampled text, and layout templates. This let them pretrain Donut on millions of synthetic documents before fine-tuning on smaller real datasets for specific tasks, which is also how they sidestepped the language-inflexibility problem that plagues OCR engines. Generate synthetic documents in a new language and you have training data for that language, no OCR retraining required.

Left Behind — by MStewartPhotography via flickr, BY 2.0.

What the numbers actually show

The headline results come from three types of tasks: document classification, information extraction, and visual question answering.

On the RVL-CDIP dataset (a standard benchmark of 400,000 scanned documents across 16 categories, like letters, forms, and invoices), Donut hit 95.30% classification accuracy. That's competitive with or better than OCR-based baselines, and it got there while running noticeably faster at inference time, since it skips the OCR pass entirely.

The information extraction results are where the "no OCR" bet pays off most visibly. On CORD, a public benchmark of Indonesian receipts with fields like menu items, prices, and totals, Donut scored a 91.6 F1, beating OCR-based baselines that used the same underlying document understanding architecture but relied on an external OCR engine's output as input. On a Japanese business card dataset built for the paper, Donut similarly outperformed OCR-pipeline baselines despite Japanese being a language where OCR engines historically underperform relative to English.

Speed is the other half of the story, and it's not a footnote. Because Donut doesn't run a separate OCR pass, inference is meaningfully faster than a comparable two-stage system, particularly for high-resolution document images where OCR itself is often the slowest part of the pipeline. The paper reports Donut processing documents in roughly half the time of the OCR-based baselines they compared against, which matters a lot if you're running this at any real volume rather than as a one-off demo.

Where it breaks: small text, weird layouts, and training data hunger

This isn't a paper claiming OCR is obsolete, and the honest limitations are worth naming.

First, Donut still struggles with dense, small text at high resolution. Fixed input resolution to a vision transformer means there's a ceiling on how much fine detail the model can resolve, and a page crammed with 8-point font pushes against that ceiling in ways a dedicated OCR engine, built specifically to resolve small glyphs, doesn't.

Second, the model needs task-specific fine-tuning to be genuinely good at anything. Pretraining gets you a model that understands "documents" in a general sense, but extracting the right fields from a specific document type still takes labeled examples of that document type. That's not unusual for document AI, but it does mean the "just point it at a new document type and go" story has an asterisk.

Third, and this one's structural: Donut generates output as a flattened sequence, and generation is inherently harder to make fully reliable than a well-designed extraction pipeline where a rule-based post-processing step can catch obviously malformed output. When Donut is wrong, it can produce plausible-looking structured output that's still wrong, and there's no OCR confidence score or bounding box to fall back on for a sanity check. That's the flip side of skipping the intermediate representation: you also lose the interpretability that intermediate representation gave you.

Why it matters if you're building or buying OCR

For anyone building document processing pipelines, Donut's real contribution isn't "delete your OCR engine." It's evidence that end-to-end, OCR-free models are viable for a meaningful slice of document understanding tasks, particularly structured extraction from documents with predictable layouts, like receipts, forms, and business cards.

If you're building on top of an OCR-and-understanding pipeline today, this suggests a few practical things worth checking. Does your downstream model actually need transcribed text, or would it do just as well (or better) working from the image directly for your specific extraction task? Is OCR latency actually a bottleneck at your volume, and would an end-to-end model's speed advantage matter to your product? And critically, does your current pipeline have any way to catch and correct OCR errors before they propagate, or are you silently trusting whatever the OCR engine hands you?

Donut itself, released alongside code, trained weights, and the SynthDoG generator, has since become a real building block in the open-source document AI ecosystem, and later models (like Nougat for scientific PDFs) borrowed its "generate structured tokens directly from pixels" approach. The bigger idea the paper puts on the table is that OCR isn't a law of nature for document understanding. It's a design choice, and for a meaningful class of documents, particularly ones with consistent layouts and known field types, skipping it outright can be both faster and more accurate than doing it well.

That said, it's not a universal replacement. If you're dealing with dense, unpredictable text at scale, a strong OCR engine paired with a good extraction model is still the safer bet today. The lesson from Donut isn't "OCR is dead." It's "check whether you actually need it for the job in front of you," which is a more useful question than most papers manage to leave you with.