Arabic Manuscripts Don't Read Top to Bottom, and Now There's a Dataset That Admits It

Picture a scribe copying a religious text by hand, centuries before anyone imagined a scanner. Partway down the page they realize a line got skipped a few paragraphs back. There's no backspace, no sticky note. So they write the missing line in the margin, small and cramped, and draw a tiny mark showing exactly where it slots back in. Multiply that by hundreds of pages and you get a manuscript where the "correct" reading order zigzags off the page and back, and any OCR system reading top to bottom in a straight line will confidently produce nonsense.

That's the exact problem a new dataset called AraMS-28k sets out to fix, and it's a genuinely useful piece of infrastructure if you care about getting text out of old documents.

Why historical Arabic manuscripts are a nightmare for OCR

Modern OCR is good at modern, printed, left-aligned text. It struggles hard the moment you introduce handwriting, and it struggles even harder with Arabic script specifically, because Arabic letters change shape depending on where they sit in a word, connect to their neighbors in a continuous flowing line, and were often written by scribes with wildly different personal styles across centuries and regions. This particular flavor of the problem has its own name: handwritten text recognition, or HTR, distinguishing it from the comparatively tame task of reading a printed page.

Now stack historical manuscripts on top of that. The paper's authors pulled together 14 real manuscript books totaling 3,043 pages and 28,600 individually annotated lines of text. Thirteen of the books are hand-copied, spanning three distinct Arabic script traditions: Naskh (a rounded, formal script still close to what you'd see in a printed Quran), Ruq'ah (a fast, simplified cursive style), and Maghrebi (the angular script tradition that developed in North Africa and Andalusia, visually distinct enough from Eastern Arabic scripts that a model trained on one can fail badly on the other). One book is a lithographed print rather than handwriting, included specifically to give the dataset some format variety.

Of the 28,600 lines, 27,971 are main-text and 629 are margin annotations. That margin number looks small next to the main-text count, but it's the part of the dataset that makes this release worth paying attention to.

The note in the margin isn't a footnote, it's a detour

Most OCR datasets treat a page as a simple top-to-bottom, left-to-right stream of lines. Real manuscripts don't work that way. Marginalia (notes written in the margins) can be genuine footnotes that stand alone, or they can be corrections meant to be read as part of the main text at a specific, precise point, exactly like our forgetful scribe above.

AraMS-28k is the first release, as far as the authors know, to annotate that distinction at the line level for a historical Arabic manuscript corpus. Every margin line gets labeled as main-text or margin, and when a margin line has an unambiguous attachment point in the main text, it also gets what the paper calls an insertion anchor: a link back to exactly where that line belongs in the reading order.

That sounds like a small annotation choice, but it's the difference between a dataset that teaches a model to transcribe isolated lines and one that teaches a model to reconstruct what the document actually says when read in order. If you're building any downstream application on manuscript OCR (search, translation, citation), the second thing is what you need. A perfect transcription of every line on a page is close to useless if the lines come back in the wrong sequence.

The vowels nobody actually wrote down

Arabic has a second wrinkle that trips up a lot of OCR and NLP work: diacritics. These are small marks above and below letters that indicate short vowels and other pronunciation details. Fully vocalized Arabic, with every diacritic marked, is unambiguous to read. The catch is that almost nobody writes that way by hand. Scribes, like most native writers of Arabic today, left diacritics out except where genuinely necessary, relying on context and vocabulary the way English readers infer "read" is past tense from context rather than needing it spelled out.

That creates a mismatch: if your reference transcriptions are fully vocalized (which makes them precise and useful as ground truth) but the manuscript images are not, a model trained naively on that pairing learns to hallucinate diacritics that were never physically on the page. AraMS-28k handles this by releasing two versions of every line's transcription: the raw, fully diacritized text, and a diacritic-normalized version stripped down to match what a scribe actually wrote. That gives anyone training a model the option to match the task to reality instead of quietly overfitting to invisible ink.

How you label 28,600 lines without losing your mind (or your accuracy)

Annotating this much historical material by hand, line by line, margin by margin, anchor by anchor, is slow and expensive if you're relying purely on human experts, and error-prone if you're relying purely on automated OCR. The authors built a pipeline called RefLAM to split the difference.

The core idea: instead of asking a multimodal large language model to transcribe a manuscript page cold and hoping for the best, RefLAM aligns the model's OCR output against independently sourced clean transcriptions of the same text. In other words, if a scholarly edition of the book already exists in typed form, you use that as a reference to check the model's guesses against, catching cases where the model misread a letter or dropped a word. Every line still gets routed through human review, so the pipeline is really automatic verification plus expert oversight, not a replacement for either. That's a sensible design: pure LLM-OCR on 700-year-old handwriting is going to make mistakes, and pure manual annotation of 28,600 lines is a multi-year project for a small team. Grounding the automated pass against an independent source of truth is what makes the combination trustworthy rather than just fast.

Naskh, Ruq'ah, Maghrebi: one dataset, a real generalization gap

The paper doesn't just release the data. It also runs baseline handwritten text recognition experiments using two open-source HTR systems: Kraken, a widely used engine in the digital humanities community for transcribing historical documents, and HATFormer, a more recent transformer-based recognition model. The point of running baselines isn't to claim a new state of the art; it's to show how hard the dataset actually is and to give future researchers a number to beat.

The most useful result here is what the authors call a cross-script generalization gradient. They tested performance on pages similar to what the model saw during training (in-distribution), then on pages from books the model has never seen at all. Accuracy drops as you move from the first case to the second, and the size of that drop tracks with how different the unseen book's script tradition is from what the model trained on. A model that's comfortable with Naskh doesn't automatically read Maghrebi well, and vice versa. That's not a surprising result if you've looked at how visually different these script traditions are, but having it quantified on a real, held-out dataset rather than assumed is exactly the kind of unglamorous, useful finding that lets people building real systems set expectations correctly instead of guessing.

Why it matters for anyone building or buying OCR

If you're evaluating OCR or document-AI tools for anything beyond clean, printed, Latin-script documents, this paper is a useful reality check on three fronts.

First, "reads the text" and "reads the text in the right order" are different capabilities, and most OCR benchmarks only test the first one. Reading order matters just as much for modern documents as it does for margin notes in a manuscript: multi-column PDFs, forms with sidebars, insurance documents with stamped annotations, contracts with handwritten amendments in the margin. A system that transcribes every character correctly but stitches them together in the wrong sequence produces output that looks plausible and is wrong. AraMS-28k's insertion-anchor annotations are a rare example of a dataset that actually measures this instead of assuming a document is one clean vertical stream.

Second, script and style variation inside a single language is a bigger deal than it sounds. "Arabic OCR" is not one problem; Naskh, Ruq'ah, and Maghrebi behave differently enough that a model tuned for one can fail on another, and the same logic applies to handwriting styles, fonts, and layouts in any language. If you're choosing between OCR engines for a product, ask what specific scripts, hands, or document styles the benchmark numbers were measured on, not just what language.

Third, ground truth itself is not free of assumptions. The vocalized-versus-unvocalized diacritic split is a reminder that "the correct transcription" depends on what you're trying to match, the idealized scholarly text or what's physically on the page. That distinction generalizes well beyond Arabic: any OCR ground truth built from a clean reference edition risks teaching a model to output the reference instead of what's actually written, and it's worth checking which one a given benchmark is really measuring.

What this dataset doesn't claim to solve

It's worth being honest about scope. Fourteen books and 3,043 pages is a meaningful release, and the largest of its kind that the authors are aware of, but it's still small next to printed-text OCR datasets that run into the millions of pages, and 629 margin lines is a modest sample for training a model to handle insertion anchors reliably on its own. The baselines reported are exactly that, baselines, not a solved benchmark; the cross-script generalization gap the authors found is itself evidence that a model trained on this data won't automatically transfer to manuscripts outside these three script traditions or outside Arabic entirely. And the CC BY-NC-SA 4.0 license means it's available for research, not for training a commercial product outright.

None of that undercuts the contribution. Good datasets rarely solve a problem outright; they make the problem measurable, which is the precondition for anyone actually solving it. AraMS-28k does that for a part of OCR that's usually ignored: the manuscripts where the text doesn't just sit there waiting to be read, it has to be reconstructed.