One Patch Is Enough: How a 2B-Parameter Model Out-Read GPT-5.4 on Street Signs

Point Claude Sonnet 4.6 at a photo of a cluttered storefront and ask it to transcribe every sign in it, with exact boundaries around each word. It scores 14.1% end-to-end accuracy on the standard Total-Text benchmark. GPT-5.4 does a little better, at 17.2%. Gemini 3.1 Flash gets into the high 50s. These are three of the most capable general-purpose models that exist right now, and on a task that sounds almost trivial (read the sign, say where it is) they get beaten badly by a model with 2 billion parameters that most people have never heard of.

That model comes from a new paper out of South China University of Technology called "One Patch Is Enough." The name is the whole pitch: instead of having a model gather up a handful of image patches to represent each word it reads, give it exactly one. The paper is a good excuse to talk about why reading text in photos is a genuinely different problem from reading a scanned page, and why throwing a bigger, smarter language model at it doesn't fix the gap.

CVS Pharmacy - Sign Restoration — by pixeljones via flickr, BY-SA 2.0.

Why reading a street sign is harder than reading a page

"Scene text spotting" is the task of finding and transcribing text that occurs naturally in a photo: storefronts, product labels, road signs, graffiti, the sticker on a fire extinguisher. It's different from document OCR in one important way. A scanned page is flat, high contrast, and laid out in neat rows. A photo of a restaurant sign is small, low contrast in places, wrapped around a logo, jammed up against three other signs, and often shot at an angle.

The model has to do two things at once and get them to line up exactly: figure out where each piece of text sits (detection) and figure out what it says (recognition). Get the boundary wrong by a few pixels and you've clipped a letter. Get the boundary right but assign it to the wrong word and the whole prediction is garbage even if the letters are correct.

Multimodal large language models (MLLMs, the GPT-4V and Gemini family of models that can look at images and talk about them) approach this by chopping an image into a grid of small tiles, called patches, and treating each patch as a token, the same kind of unit the model uses for words. To point at something in the image, the model can either output raw coordinates as text, or point at one of these patch tokens directly. Pointing at a patch is more reliable, because it ties the model's answer to something it actually looked at, rather than having it guess four numbers out of thin air.

The old approach: throw several patches at each word

Once you're pointing at patches instead of typing coordinates, a reasonable next move is to have the model select several patches per word, on the theory that more evidence means a more confident answer. That's what prior methods did, and the SPaTS paper argues it backfires specifically for text.

Their Figure 1 shows why with a real sign reading something like "MANOEI-HEATHER, PLEASE, FOREVER." With multiple patches assigned per word, three failure types show up constantly: a patch lands on irrelevant background instead of the text, several patches get assigned to the same word so it's unclear which one is authoritative, and patches from neighboring words blur together into a mixed, ambiguous signal. Small, dense, curved text is exactly the situation where nearby patches are most likely to step on each other.

The fix: pick exactly one patch, then let the rest of the image help with the shape

The paper's core idea, Single-Patch Text Spotting (SPaTS), routes every text instance through a single anchor patch rather than a cluster. That anchor establishes identity: this patch is where the word "FOREVER" lives. But the anchor alone doesn't carry enough information to draw an accurate boundary around curved or irregular text, so SPaTS splits the job in two.

A component called Patch-Enhanced Decoding (PED) takes the one selected anchor, combines it with the model's language state, and then cross-attends over the entire image's feature map, not just the anchor patch, to regress the final geometry: a bounding box, a Bézier curve for text that curves around a logo, or a set of points. Think of it like a witness picking a face out of a lineup and then a sketch artist redrawing the full portrait by looking at the whole photo, not just the pointing finger. One token says what and roughly where; the rest of the image supplies the precise shape.

Teaching a model which patch to pick, without ever telling it the right answer

Here's the awkward part: no dataset says "patch #4,281 is the correct anchor for this word." Ground truth gives you a bounding box and a transcription, not a ranking of which individual patch best represents that instance. Picking the anchor is a discrete decision with no direct label to learn from.

The authors solve this with reinforcement learning, in a framework they call Single-Patch Selective Optimization (SPaSO), built on GRPO (Group Relative Policy Optimization), the same recipe DeepSeek used to train math reasoning into its models. The model samples several candidate patch choices, and whichever choices lead to better outcomes get reinforced relative to the others in the group. No separate critic network required.

Two rewards do the work. The Patch Generation Reward checks the final result: did selecting this patch lead to a correct transcription with good geometric overlap against ground truth? The Patch Dominance Reward is the more interesting one. Because the model will rarely stumble onto the perfect answer early in training, that primary reward is often zero, which gives the model nothing to learn from. The dominance reward gives partial credit just for ranking the correct patch highly among its candidates, even when the final prediction still misses. It's the difference between only rewarding a bullseye and also rewarding "getting closer to the board."

Two smaller ideas doing more work than their names suggest

Two supporting pieces round out the architecture.

Directional Embedding Alignment (DEA) addresses a subtle failure mode: patches are represented as vectors, and vectors with large magnitude can dominate the selection process even when a smaller vector is actually the better semantic match. DEA splits each patch vector into magnitude (how strong the signal is) and direction (what it actually represents), and after a warmup period it locks the magnitude, forcing selection to depend on directional alignment rather than which patch happens to be "loudest." In their ablation, removing DEA caused selection to be dominated by feature norms instead of semantic relevance.

Removing PED, and decoding geometry from the language model's internal state alone instead of re-grounding in the selected image patch, caused what the authors describe as a collapse in accuracy. That's a useful negative result: it shows the model will start hallucinating plausible-looking text locations if you don't force it to keep checking its answer against the actual pixels.

The numbers

Results are reported across three benchmarks: Total-Text (curved and multi-oriented text), CTW1500 (also curved, denser), and ICDAR2015 (real street-scene photos, generally the hardest of the three). SPaTS comes in two sizes, 2B and 4B parameters, and is compared against three tiers of competitor: frontier closed-source models (Claude Sonnet 4.6, GPT-5.4, Gemini 3.1 Flash, Seed 2.0 Pro), large open-weight models (Qwen3.5, a 397B mixture-of-experts model; InternVL3.5 at 241B; GLM-4.6V; Kimi K2.5), and specialized OCR-focused MLLMs (HunyuanOCR, PaddleOCR-VL-1.5, dots.ocr).

On Total-Text, SPaTS-4B reaches 64.9% end-to-end F-score under the strict no-dictionary setting and 74.9% with a lexicon to help disambiguate words, against Claude Sonnet 4.6's 14.1% and 15.9%. The closed-source frontier models simply aren't competitive here, regardless of how good they are at everything else. On CTW1500, SPaTS-4B's end-to-end score reaches 79.6% under the full-lexicon protocol. On ICDAR2015, the paper states plainly that SPaTS lifts end-to-end accuracy by roughly 50% relative to the prior best result.

A direct ablation is worth calling out because it isolates the paper's central claim from everything else going on in the architecture: on the same backbone, they compared coordinate-text output, multi-patch grounding, and single-patch grounding against each other. Single-patch won outright. That's good evidence the gain is coming from the idea in the title, not just from a bigger training run or a better backbone.

The reinforcement learning stage matters more than you might expect. Going from supervised fine-tuning alone to adding SPaSO produced what the authors call a 20.7-point jump in end-to-end F-measure on CTW1500 under the full-lexicon setting, from 58.9% to 79.6%. Supervised training gets a model that transcribes characters accurately; the RL stage is what teaches it to actually pick good anchors.

Why it matters for anyone building or choosing OCR

The most useful takeaway here isn't the specific architecture, it's what the comparison table says about the state of general-purpose models. Claude Sonnet 4.6, GPT-5.4, and Gemini 3.1 Flash are excellent at reasoning, coding, and holding a conversation about an image. They are currently bad at precise, dense text localization, scoring in the teens and twenties on a task a purpose-built 2B model handles at 60 to 80%. If your product needs someone to draw a tight box around every word in a photo of a receipt or a cluttered shelf, a general chat model is the wrong tool, no matter how new or expensive it is.

The second takeaway is that parameter count is a weak signal for spotting quality. A 2B model beat a 397B mixture-of-experts model and a 241B model on this task. If a vendor tells you their OCR is good because it's "powered by" a huge frontier model, that claim says very little about how it'll do on a curved logo or a blurry storefront photo. Ask for accuracy numbers on messy real-world images specifically, not on clean scanned documents, because those are two different problems with two different difficulty curves.

The third takeaway is really an argument for not betting on a single engine at all. The right model for a clean invoice scan is not necessarily the right model for a warped receipt or a photo of a street sign, and this paper is a clean illustration of just how wide that gap can be between models that all claim to "read images." Running a document through multiple engines and picking the best result costs less than assuming one giant model will handle every case well.

One caveat worth sitting with: everything here is Latin-script scene text, no handwriting, no dense forms, no non-Latin scripts, no genuine paper documents. It's a strong result on the problem it was tested on. Whether the same single-anchor idea holds up on a stack of invoices or a handwritten note is a different question the paper doesn't answer.

The rough edges nobody's papering over

The authors are upfront about two real limitations. First, the RL stage adds meaningful training overhead and depends on carefully tuned reward shaping to stay stable. This isn't a drop-in efficiency win over supervised training, it's a more complicated recipe that trades training simplicity for accuracy.

Second, DEA locks the magnitude branch of its patch representations after a warmup period, and the authors flag that this could hurt robustness under extreme lighting changes, where a patch's raw feature scale might actually carry useful information that the frozen model is now ignoring. That's a specific, plausible failure mode, not a vague disclaimer.

There's also the practical matter that code is only "to be released soon," so right now this is a paper, not something you can install and test against your own documents. Worth watching for when it ships, especially given how far ahead of both frontier chat models and dedicated OCR systems it claims to be, but for now the numbers exist only in the tables above.