An old-school OCR pipeline just beat Gemini, GPT and Claude at reading Sanskrit

Picture a 400-year-old Sanskrit manuscript. The scribe crammed commentary into the margins, wrote some passages in a spiral around a devotional illustration, and packed 44 lines onto a single page in the densest sections. Someone trained to read the script might spend a full minute just working out which stroke belongs to which line before transcribing a single word. Here's the strange part: a fairly conventional, three-stage OCR pipeline, the kind most computer vision people would call dated, ends up beating Gemini, GPT and Claude at reading these pages, once a human has hand-corrected exactly three of them.

That's the finding from a new paper, "Impact of Iterative Fine-Tuning on Transcription Accuracy in Complex Historical Sanskrit Manuscripts," by Kartik Chincholikar, Kaushik Gopalan and Mihir Hasabnis. It's a useful reality check for anyone who assumes the answer to a hard document problem is always to point a bigger multimodal model at it.

Why these pages break the usual OCR playbook

Most OCR systems assume text runs in neat horizontal rows down a page, the way a printed book or a receipt does. Historical Sanskrit manuscripts break that assumption in several directions at once. The researchers group their three sample manuscripts by layout difficulty: a moderate layout with marginal notes and interlinear glosses, a dense layout averaging 44.43 text lines per page, and a circular layout where script wraps around illustrations instead of running straight.

On top of that geometry problem, there's an appearance problem. Every scribe has their own handwriting, shaped by period and region. Pages carry stains, tears and uneven lighting from being photographed in an archive rather than run through a flatbed scanner. And correcting any of this requires someone who can actually read historical Sanskrit, a skill in short supply and expensive to hire. That scarcity is the real constraint the paper is designed around: cutting how much correction a domain expert has to do, not just improving raw accuracy.

Three models bolted together, not one giant network

The pipeline the authors build has three separate stages instead of one end-to-end network.

First, a frozen, pretrained character detector called CRAFT scans the page and produces a heatmap of where individual characters likely sit, similar to a thermal image showing which patches of the page contain ink shaped like a letter. Second, a graph neural network built on an architecture called SplineCNN takes those detected characters and decides, one pair at a time, whether two characters belong on the same line. Think of it as connect-the-dots: each character is a dot, and the network's job is figuring out which dots chain together into a line of text, in what order. Third, a CNN-BiLSTM-CTC network reads each assembled line and converts it into Unicode text. That last piece uses a training technique called CTC (connectionist temporal classification), which lets the model output a transcription without needing to know in advance exactly where each letter starts and ends on the image, useful when handwriting doesn't segment cleanly.

For the circular manuscripts, the pipeline adds a geometric trick: it finds a local coordinate system that follows the curve of the text and effectively unrolls it into a straight strip before handing it to the reader. Without that step, a recognizer trained on straight lines would have no chance with text that bends around an illustration.

Two dials, tuned separately

The paper's central idea is fine-tuning this pipeline on two dials that turn independently.

Layout-level fine-tuning retrains the graph network on a page where a human has corrected the line groupings, using data augmentation (warping, skewing, and randomly dropping nodes) to stretch a single corrected page into a much larger training signal, then running ten epochs with a loss function tuned to focus on the hardest examples. Appearance-level fine-tuning retrains the recognizer on a page where a human has corrected the actual transcribed text, adapting it to that specific scribe's handwriting and that specific page's wear and tear over 60 iterations.

Because these two dials are separate, correcting one doesn't force you to retrain the other. You can teach the system "this scribe writes marginal notes in a column on the left" without touching what it knows about that scribe's letterforms, and vice versa.

The loop itself is straightforward: correct a page, fine-tune both models on that correction, then run the improved pipeline on the next page. Since the model now has a working model of this scribe's layout habits and handwriting, the next page needs less correction. Repeat that a few times and the correction burden drops fast, which is the actual point: the paper isn't chasing a leaderboard number, it's trying to get a historian's afternoon back.

Three pages, three manuscripts, real numbers

The team built their evaluation set from three Sanskrit manuscripts: a moderate-layout one (15 pages, averaging 21.07 lines per page), a dense-layout one (7 pages, averaging 44.43 lines per page), and a circular-layout one (9 pages, averaging 28.33 lines per page), 31 pages in total. Each page carries granular layout annotations plus a full Unicode transcription, encoded in PAGE-XML, a format archives and libraries already use for this kind of ground truth. That alone is a useful contribution: annotated historical-manuscript datasets at this level of detail are rare, and Sanskrit is underserved even by the low bar OCR research usually clears for non-Latin scripts.

On accuracy, after fine-tuning on just three pages, running fully automatically (meaning the loop uses the model's own predictions as pseudo-corrections rather than a human's), the moderate-layout manuscript saw its TextEdit score drop by 34.1% and its page-level character error rate (Page-CER) drop by 37.5%. The dense layout improved 22.9% on both metrics. The circular layout, the hardest case, improved 15.5% on TextEdit and 8.1% on Page-CER.

TextEdit is roughly the share of characters you'd need to insert, delete or swap to turn the model's output into the correct transcription. Page-CER measures the same kind of error but across the whole page rather than line by line, which matters here specifically because a model that gets the reading order wrong can still score fine on a per-line metric while producing a scrambled page.

When a human actually corrects the layout as part of the loop, rather than letting it run fully automatically, the gains get bigger: the moderate-layout manuscript's TextEdit score fell 51.3%, and the dense layout improved 36.2% on TextEdit and 36.9% on Page-CER.

What makes the loop worth running is the cost side. Manually correcting a page took 38.0 seconds for the moderate layout, 165.5 seconds for the dense layout, and 109.0 seconds for the circular layout, all comfortably under three minutes even in the worst case. Fine-tuning the two models between pages cost 41.10 seconds for the graph network and 27.51 seconds for the recognizer, under 70 seconds combined. That compute cost is smaller than the correction time it saves on the very next page, so the loop pays for itself almost immediately rather than requiring some large upfront investment before it's worth it.

Why it matters if you're building or choosing OCR

The lesson here isn't "go implement CRAFT and SplineCNN." It's that a narrow pipeline you can cheaply fine-tune on a handful of your own documents can outperform a general-purpose model that's never seen your document type before, and it doesn't take much fine-tuning to get there. This holds especially when your documents share a strong internal pattern: one scribe's handwriting, one archive's camera setup, one vendor's invoice template, one court's filing format.

If you're processing a large batch of visually similar documents, the economics tilt toward spending a couple of hours correcting a few sample pages and fine-tuning on them, instead of paying per page to send every document to a frontier model that's guessing from zero examples of your specific layout. That calculus flips back the other way when your documents are genuinely heterogeneous and there's no shared pattern for a small model to learn. Know which situation you're actually in before picking a strategy, because the wrong call either way is expensive: overbuilding a fine-tuning loop for a one-off batch wastes engineering time, and underbuilding one for a recurring, high-volume document type quietly bleeds accuracy and API cost forever.

Where the "smart" models actually struggled

The paper also benchmarks four multimodal large language models straight out of the box: Gemini 3.5 Flash, GPT-5.6 Terra, Claude Sonnet 5, and Sarvam Vision, a model built specifically for Indic scripts.

On the moderate-layout manuscript, Gemini 3.5 Flash came out on top of that group at 0.26 TextEdit and 0.35 Page-CER, with Claude Sonnet 5 at 0.60 and 0.64, and GPT-5.6 Terra at 0.71 and 0.74, meaning roughly seven out of every ten characters wrong. On the dense layout, Gemini scored 0.30 and 0.44 while Sarvam Vision came in at 0.43 and 0.64. On the circular layout, Gemini again led the pack at 0.51 on both metrics, while Sarvam Vision reportedly went into what the authors call runaway generation: it kept producing text well past where the actual passage ended, effectively inventing manuscript content that isn't on the page.

Then comes the punchline. After fine-tuning on just three pages, the traditional pipeline beat Gemini 3.5 Flash, the best of the four multimodal models, on all three manuscripts, on both metrics. General-purpose vision-language models trained on enormous volumes of text and images still aren't great at cold-reading a specific historical script they've likely seen little of during training. A handful of corrected examples closed a gap that raw scale hadn't.

What the paper doesn't answer yet

Thirty-one pages across three manuscripts is a small evaluation set, though it's worth remembering why: annotating each page requires genuine paleography training, so this isn't a dataset you grow by paying crowdworkers. The character detector at the front of the pipeline, CRAFT, stayed frozen throughout. Only the graph network and the recognizer got fine-tuned, so the whole system is still bottlenecked on how well that untouched first stage locates characters on pages with unusual lighting or heavy staining.

There's also a real setup cost. Building and fine-tuning a character detector, a graph network and a CNN-BiLSTM-CTC recognizer is a heavier lift than pointing a script at a vision-language model's API. It only pays off once you have enough volume to amortize that setup. A handful of pages, transcribed once and never again, is probably still faster to hand-correct or send to the best MLLM you can reach.

The code and dataset are public on GitHub under flame-cai/gnn-synthetic-layout-historical, PAGE-XML annotations included. That's a genuinely useful thing to have released. Digitized historical-manuscript datasets with this level of layout ground truth are hard to come by, and whatever gets built on top of this pipeline next, the dataset is likely to outlast it.