Synthetic Layouts, Better Vertical Japanese OCR: Inside Synth-JDoc
By Gregory Hain
Based on: Synth-JDoc: Synthesizing a Japanese Document Image Dataset for OCR with Diverse Layouts and Embedded Images — Keito Sasagawa, Shuhei Kurita, Daisuke Kawahara
Drop a multi-column Japanese government document into a Large Vision Language Model (LVLM), ask it to transcribe the page, and it may fail in a very specific way. In one example from the Synth-JDoc paper, a model reads section “A1,” skips the text in the next vertical column, and jumps directly to “Q2.”
Japanese documents can combine horizontal text (yokogaki) and vertical text (tategaki). In vertical Japanese text, characters run from top to bottom and lines run from right to left. The paper notes that vertically written Japanese remains underrepresented in OCR training data, and that current LVLMs perform considerably worse on it than on horizontal Japanese text.
To address that gap, Keito Sasagawa, Shuhei Kurita, and Daisuke Kawahara developed Synth-JDoc. Their paper, “Synth-JDoc: Synthesizing a Japanese Document Image Dataset for OCR with Diverse Layouts and Embedded Images”, describes a synthetic-data pipeline built around HTML/CSS rendering, text-to-image generation for embedded figures, and image degradation.
The final dataset contains 17,970 curated document images. In evaluations on real-world vertically written Japanese documents, models fine-tuned on Synth-JDoc achieved the best reported scores under both evaluation settings for four of the five model families tested. The project code is open-sourced on GitHub (llm-jp/synth-jdoc), with the dataset hosted on Hugging Face (llm-jp/Synth-JDoc).
Here is a technical teardown of the vertical Japanese OCR problem, how Synth-JDoc builds its synthetic pages, and what its experiments show.
Why Vertical Japanese Is Difficult for LVLMs
Document transcription requires more than recognizing individual characters. A model must also follow the page’s reading order.
Japanese vertical documents add a particular challenge:
- Vertical Directionality: Characters in a vertical line are read from top to bottom, while lines proceed from right to left.
- Multi-Column Layouts: The paper describes multi-column reading order as progressing sequentially from the uppermost column to the lowest.
- Mixed Writing Styles: Japanese documents may contain vertical and horizontal writing on the same page.
The paper’s motivating example is a real document page where the original Qwen3-VL model omitted the second column from the top. The fine-tuned model did not skip that text. The authors argue that a Japanese OCR dataset with vertical text, multi-column layouts, and inserted images is needed to improve this type of document reading.
Building that dataset from real documents is expensive. Manual annotation does not scale easily, while extracting training text with an OCR system introduces recognition errors and requires collecting source document images. Existing synthetic approaches also have limits: SynthDoG can generate Japanese document images but produces less realistic layouts, while JSSODa supports vertical and horizontal text in one- to four-column layouts but uses black text on a white background without inserted images.
The Synth-JDoc Architecture: Code-Driven Synthesis
Synth-JDoc uses a three-stage pipeline: document element preparation, document image synthesis, and noise application.
+-------------------------------------------------------------------------+
| Stage 1: Document Element Preparation |
| - Text sourcing: JSSODa corpus (JUMAN dictionary nouns -> llm-jp-3.1) |
| - Paragraph segmentation and title generation |
| - Linear Assignment Problem matching paragraphs to image placeholders |
| - Image prompt generation (Qwen3-30B) -> Z-Image-Turbo |
| - Caption generation (Qwen3-VL-30B) at 9:1 short-to-long ratio |
+-------------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------------+
| Stage 2: HTML/CSS Document Rendering |
| - 8 layout combinations: Horizontal/Vertical x 1, 2, 3, 4 Columns |
| - 49 Japanese Google Fonts |
| - 25% title presence probability; single-column or spanning figures |
| - Web-browser rendering |
+-------------------------------------------------------------------------+
|
v
+-------------------------------------------------------------------------+
| Stage 3: Noise Pipeline (17,970 Curated Images) |
| - 40% clean programmatic renders |
| - 30% scan-like noise |
| - 30% Augraphy degradation |
+-------------------------------------------------------------------------+
1. Document Element Preparation
The pipeline begins with text from JSSODa. That dataset’s text was generated by extracting nouns from the JUMAN dictionary and prompting llm-jp-3.1-13b-instruct4 to generate sentences related to each noun. Synth-JDoc uses the same model to generate a title for each text.
Next, the text is split into paragraphs at double newlines (\n\n). To create illustrated documents, the pipeline inserts image placeholders:
- The number of images per document is chosen randomly, from zero up to half the number of paragraphs.
- Placeholders are inserted randomly before or after paragraphs.
- The system pairs placeholders with paragraphs by minimizing the sum of squared distances between them through a linear assignment problem.
For each paired placeholder, Qwen3-30B-A3B-Instruct-2507 produces an image-generation prompt based on the associated paragraph. Z-Image-Turbo then generates the image. Finally, Qwen3-VL-30B-A3B-Instruct generates an image caption. Concise captions and longer, multi-sentence captions are used at a 9:1 ratio.
2. Browser-Based Layout Synthesis
With text, titles, images, and captions prepared, Synth-JDoc builds HTML/CSS pages and renders them as images in a web browser.
- Layout Combinations: The dataset is distributed uniformly across eight combinations: horizontal or vertical writing, each with one through four columns.
- Typographic Diversity: Fonts are selected randomly from 49 Japanese options available through Google Fonts.
- Layout Variation: Titles appear with a 25% probability. In layouts with two or more columns, images may remain within one column or span all columns. Figure-caption styles, including “Figure N:” and “Fig. N:”, are also randomized.
Because the text is rendered directly into the synthetic documents, the pipeline avoids the OCR recognition errors that arise when labels are extracted from existing document images.
3. Noise and Scanner Simulation
The final training set combines clean synthetic images, scan-like images, and Augraphy-processed images.
- Clean Renders (40%): Synthetic documents before noise application.
- Scan-Like Noise (30%): Gaussian noise, rotation, perspective transforms, vertical and horizontal shadows, vignette effects, and occasional global blur.
- Augraphy Processing (30%): A document-degradation pipeline with three stages:
- Ink Phase:
InkBleedandInkMottling. - Paper Phase: Added color and textural patterns.
- Post Phase: Further degradation such as stains, scribbles, shadows, and simulated folding.
- Ink Phase:
After manually filtering images with synthesis errors or potential copyright concerns, the final Synth-JDoc training set contains 17,970 images.
Experimental Results: The VJRODa Benchmark
The researchers evaluated the models on VJRODa, a dataset of 100 real-world vertically written Japanese document images and corresponding texts. The paper describes VJRODa as constructed from real PDF pages; its limitations section notes that the PDFs were published by government agencies and are relatively clean.
Five open-source LVLMs were fully fine-tuned, with all module parameters updated:
Qwen2.5-VL-7B-InstructQwen3-VL-8B-InstructInternVL3-8B-hfInternVL3.5-VL-8B-hfGemma 3 12B IT
Training used a batch size of 32, the AdamW optimizer, and a learning rate of 2e-05. The prompt asked each model to output all text in the image in standard Japanese reading order. The paper kept the training configuration and training-data volume consistent across the fine-tuning settings.
Evaluation Metrics and the Repetition Trap
The authors report two metrics:
- Character Error Rate (CER $\downarrow$): Edit distance between a model output and the ground-truth text, divided by the number of ground-truth characters and multiplied by 100. Lower is better.
- SacreBLEU (BLEU $\uparrow$): Character-level BLEU after character-level tokenization. Higher is better.
- Text Preprocessing: Unicode NFKC normalization and whitespace removal before scoring.
The results are reported under two output settings:
- Raw Output: Scores calculated on the model output as generated.
- Remove Repetition: Scores calculated after removing trailing repetitive strings.
The second setting matters because LVLMs can repeat the same string. The paper uses both settings to separate changes in repetitive generation from changes in core character-recognition performance.
Performance Breakdown
| Model | Training Dataset | Raw CER ($\downarrow$) | Raw BLEU ($\uparrow$) | Rep-Removed CER ($\downarrow$) | Rep-Removed BLEU ($\uparrow$) |
|---|---|---|---|---|---|
| Qwen2.5-VL-7B | Zero-Shot Base | 154.0 | 20.1 | 88.5 | 22.0 |
| + JSSODa | 65.1 | 51.5 | 40.5 | 61.1 | |
| + Nano Banana Pro | 161.0 | 20.9 | 135.0 | 24.4 | |
| + Synth-JDoc (Ours) | 34.5 | 66.8 | 32.0 | 69.5 | |
| Qwen3-VL-8B | Zero-Shot Base | 116.0 | 32.6 | 45.6 | 52.5 |
| + JSSODa | 130.0 | 29.9 | 65.5 | 49.4 | |
| + Nano Banana Pro | 177.0 | 16.1 | 138.0 | 17.3 | |
| + Synth-JDoc (Ours) | 43.9 | 57.4 | 25.0 | 70.8 | |
| InternVL3-8B | Zero-Shot Base | 121.0 | 26.0 | 66.5 | 40.8 |
| + JSSODa | 251.0 | 26.1 | 73.5 | 54.9 | |
| + Nano Banana Pro | 173.0 | 15.3 | 140.0 | 19.3 | |
| + Synth-JDoc (Ours) | 70.9 | 47.8 | 38.9 | 68.1 | |
| InternVL3.5-VL-8B | Zero-Shot Base | 121.0 | 29.2 | 56.1 | 41.0 |
| + JSSODa | 57.9 | 62.3 | 37.2 | 71.9 | |
| + Nano Banana Pro | 173.0 | 15.6 | 117.0 | 18.8 | |
| + Synth-JDoc (Ours) | 36.8 | 66.7 | 25.9 | 78.3 | |
| Gemma 3 12B IT | Zero-Shot Base | 125.0 | 17.5 | 67.9 | 23.3 |
| + JSSODa | 77.6 | 27.9 | 67.4 | 27.2 | |
| + Nano Banana Pro | 196.0 | 8.5 | 145.0 | 9.3 | |
| + Synth-JDoc (Ours) | 128.0 | 18.7 | 96.3 | 26.5 |
For Qwen2.5-VL, Qwen3-VL, InternVL3, and InternVL3.5, the Synth-JDoc fine-tuning row has the best CER and BLEU scores under both Raw Output and Remove Repetition.
- On Qwen3-VL-8B, repetition-removed CER fell from 45.6 to 25.0, while BLEU rose from 52.5 to 70.8.
- On InternVL3.5-8B, repetition-removed CER fell from 56.1 to 25.9, while BLEU rose to 78.3.
- Fine-tuning on JSSODa improved some models but did not improve every setting. For example, Qwen3-VL’s raw CER rose from 116.0 to 130.0.
Real-World Case Study
The paper’s qualitative example uses a document concerning National Pension notices (Nenkin Tokubetsubin).
The base Qwen3-VL-8B model produced “A1” in the first section and then skipped the text in the second column from the top before continuing at “Q2” in the third. Its CER was 47.4. The Synth-JDoc fine-tuned model did not skip that column, and its CER was 3.84.
Two Instructive Experimental Results
The paper also tests two important boundaries of the approach.
1. The Nano Banana Pro Baseline
The researchers generated a baseline dataset using Nano Banana Pro (gemini-3-pro-image-preview). Prompts specified the same text, titles, figure captions, writing direction, and number of columns used for Synth-JDoc. When captions were present, the prompts also requested corresponding figures.
The images could look realistic, with details such as wooden desks and page folds. But the paper reports several document-generation failures:
- Distorted Characters: Generated Japanese characters could be malformed.
- Text Mismatch: The image text could differ from the prompt.
- Vertical Layout Failure: The model did not reliably create multi-column documents with vertically written text. In one example, a prompt requesting a vertically written two-column layout produced two separate vertically written pages instead.
The Nano Banana Pro fine-tuning setting generally degraded results relative to the original models. The paper notes one exception: Qwen2.5-VL’s BLEU improved slightly, though its CER worsened substantially.
2. The Gemma 3 Resolution Constraint
Synth-JDoc did not improve Gemma 3. The authors attribute this result to how Gemma 3 handles input images.
Qwen2.5-VL and Qwen3-VL use Native Dynamic Resolution, while InternVL3 and InternVL3.5 use a Dynamic Tiling Strategy. These approaches can process images with different aspect ratios and resolutions. By contrast, the paper states that Gemma 3 resizes input images to a fixed 1:1 resolution.
Synth-JDoc contains document images with a range of resolutions. The authors therefore hypothesize that Gemma 3’s fixed resizing prevented it from learning effectively from the dataset. They present this as evidence that OCR performance depends on both dataset design and model architecture.
Ablation Study: Images and Captions
The authors also tested the effect of embedded images and their captions. Before applying noise, they masked the images, the captions, or both with the background color, then trained Qwen3-VL-8B and InternVL3.5-VL-8B on those variants.
| Ablation Setting (Qwen3-VL-8B) | Raw CER ($\downarrow$) | Raw BLEU ($\uparrow$) | Rep-Removed CER ($\downarrow$) | Rep-Removed BLEU ($\uparrow$) |
|---|---|---|---|---|
| Full Synth-JDoc | 43.9 | 57.4 | 25.0 | 70.8 |
Masked Images (w/o image) |
60.5 | 48.1 | 36.8 | 67.4 |
Masked Captions (w/o caption) |
120.0 | 30.2 | 51.2 | 54.5 |
Masked Both (w/o image + caption) |
55.7 | 48.9 | 39.3 | 64.8 |
| Ablation Setting (InternVL3.5-8B) | Raw CER ($\downarrow$) | Raw BLEU ($\uparrow$) | Rep-Removed CER ($\downarrow$) | Rep-Removed BLEU ($\uparrow$) |
|---|---|---|---|---|
| Full Synth-JDoc | 36.8 | 66.7 | 25.9 | 78.3 |
Masked Images (w/o image) |
45.5 | 58.8 | 38.6 | 65.2 |
Masked Captions (w/o caption) |
54.3 | 57.9 | 39.9 | 72.7 |
Masked Both (w/o image + caption) |
59.3 | 57.6 | 47.1 | 66.4 |
Removing both images and captions raised Qwen3-VL’s repetition-removed CER from 25.0 to 39.3. For InternVL3.5, it rose from 25.9 to 47.1.
For both models, the full Synth-JDoc dataset outperformed the variants with masked images or captions. The authors attribute this to synthetic pages with both elements representing more visually diverse and realistic documents.
Limitations and Future Work
The paper identifies several open constraints:
- Layout Diversity: Synth-JDoc covers horizontal and vertical layouts with one to four columns, but real documents such as newspapers can have more complex reading orders.
- Structured Elements: The authors identify graphs and tables as future targets for the synthesis pipeline.
- Decoder Repetition: Greedy decoding still produced repetitive output in some cases. The authors suggest that decoding hyperparameters may help suppress this behavior.
- Test Set Diversity: VJRODa consists of relatively clean government PDF images, so it cannot fully measure the value of the pipeline’s noise augmentation.
- NSFW Filtering: The construction pipeline does not include NSFW-content filtering. The authors note that this could be introduced during text preparation, prompt generation, image generation, or image captioning, and that both text and images should be filtered at each stage.
Why It Matters for Document AI Teams
For teams working on Japanese document OCR, the paper offers several evidence-based lessons.
1. Programmatic Layout Control Preserves the Intended Text
Synth-JDoc renders prepared text directly through HTML and CSS. That avoids OCR-derived label errors and gives the authors explicit control over writing direction, columns, titles, fonts, figure placement, and caption styles.
The Nano Banana Pro comparison does not establish that all text-to-image methods are unsuitable for OCR-data generation. It does show that, in this experiment, a high-performance text-to-image baseline struggled with exact characters and vertically written multi-column layouts.
2. Images and Captions Changed the Training Result
The ablation study found that masking embedded images or captions reduced performance for both Qwen3-VL-8B and InternVL3.5-VL-8B. In this dataset, those elements improved the results relative to the corresponding masked variants.
3. Dataset Design and Model Architecture Interact
The paper’s Gemma 3 result is a useful caution. The authors hypothesize that fixed 1:1 image resizing made it harder for that model to benefit from Synth-JDoc’s varied document resolutions, while models with Native Dynamic Resolution or Dynamic Tiling improved substantially.
By releasing code and data, Sasagawa, Kurita, and Kawahara provide a public resource for training and studying OCR on vertically written Japanese document images.