Skip to content

About

Multi-engine ATR for multiple languages and scripts

Resources

Stars

0 stars

Watchers

0 watching

Forks

Β 
Β 

Latest commit

Β 

History

172 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Multi-Engine HTR Training & Comparison Tool

A comprehensive toolkit for training and comparing different Handwritten Text Recognition (HTR) engines on historical manuscript datasets. Supports TrOCR, CRNN-CTC, Qwen3-VL, LightOnOCR, Party, and Kraken engines through a browser-based web interface (plus a PyQt6 desktop GUI and a batch CLI).

Primary Focus: Cyrillic manuscripts (Russian, Ukrainian, Church Slavonic, Glagolitic)


🎯 Features

Multiple HTR Engines

  • TrOCR: Transformer-based OCR (line-level)
  • CRNN-CTC: Puigcerver CRNN with CTC decoding (line-level, PyLaia-inspired)
  • Qwen3-VL: Vision-Language Model (line/page-level, custom prompts)
  • LightOnOCR: Lightweight VLM (~4GB VRAM, line-level, fine-tuned variants)
  • Churro: Qwen fork, experimental (line/page-level, custom prompts)
  • Party: Transformer-based HTR (line-level, multilingual)
  • Kraken: Segmentation & recognition
  • PaddleOCR: Multi-language printed/mixed text detection + recognition (whole-page, subprocess-isolated)

Commercial & Local Vision Models

  • Commercial APIs: Google Gemini, Anthropic Claude Vision (via API keys)
  • Local LLMs: OpenWebUI integration for local vision models
  • Unified interface: All models accessible through same engine plugin system

Core Capabilities

  • Web interface: Upload, segment, transcribe, correct and export in the browser; runs locally or on a server
  • Engine comparison: Compare engines side-by-side (web interface and desktop GUI)
  • Model management: Easy switching between trained models and API providers
  • Export formats: TXT, CSV, PAGE XML

Training Pipelines (GPU required)

  • CRNN-CTC: Custom CRNN training with PAGE XML support
  • TrOCR: Fine-tuning pipeline with image caching (10-50x faster)
  • Data preparation: Transkribus PAGE XML parser

Key Capabilities

  • Line segmentation (automatic or PAGE XML-based)
  • Custom prompt support (Qwen3-VL)
  • Batch processing
  • PAGE XML import/export

πŸš€ Quick Start

1. Installation

# Clone repository
git clone https://github.com/achimrabus/polyscriptor.git
cd polyscriptor

# Create virtual environment
# Requires Python 3.10–3.12 (Python 3.13+ may lack prebuilt wheels for some
# packages; check your version with: python --version)
python3 -m venv htr_env
source htr_env/bin/activate  # Linux/Mac
# or: htr_env\Scripts\activate  # Windows

Install everything in one pip call into the fresh environment. requirements-kraken.txt adds Kraken segmentation; it is optional but recommended (without it, the fast but much weaker HPP segmentation is used).

GPU install (CUDA 12.8 β€” Linux/Windows with NVIDIA GPU):

pip install -r requirements-gpu.txt -r requirements.txt -r requirements-kraken.txt \
    --extra-index-url https://download.pytorch.org/whl/cu128

CPU-only install (no GPU required):

# Linux/Mac:
pip install -r requirements.txt -r requirements-kraken.txt

# Windows (avoids a torch DLL load error on CPU-only machines):
pip install -r requirements.txt -r requirements-kraken.txt --extra-index-url https://download.pytorch.org/whl/cpu

Do not add Kraken to an existing environment later (pip install kraken or pip install -r requirements-kraken.txt as a second step). Kraken pins several shared packages tightly; installed afterwards, pip downgrades them and transformers stops importing. If you need Kraken in an environment that was set up without it, create a new environment and use the one-call install above.

2. Start the web interface

source htr_env/bin/activate        # Windows: htr_env\Scripts\activate
uvicorn web.polyscriptor_server:app --host 0.0.0.0 --port 8765

Open http://localhost:8765 in your browser. On a remote server, tunnel the port from your laptop with ssh -L 8765:localhost:8765 user@server and open the same address locally. No display, X11 or Qt is needed. Details: Web Interface.

Alternative: desktop GUI (PyQt6)

python3 transcription_gui_plugin.py      # single pages
python3 polyscriptor_batch_gui.py        # folders

Linux: PyQt6 (Qt β‰₯ 6.5) needs the system library libxcb-cursor0. If the GUI fails with "Could not load the Qt platform plugin "xcb"", install it: sudo apt install libxcb-cursor0 (Debian/Ubuntu).

For many pages: batch CLI

python3 batch_processing.py \
    --input-folder HTR_Images/my_folder \
    --engine crnn-ctc \
    --model-path models/crnn_ctc_model/best_model.pt \
    --segmentation-method kraken-blla \
    --use-pagexml

3. Train a Model (CLI, CRNN-CTC Example)

# Step 1: Parse PAGE XML export β†’ CSV format
# Works with Transkribus exports (namespace 2013-07-15) and
# eScriptorium/Kraken exports (namespace 2019-07-15) β€” both are auto-detected.
python3 transkribus_parser.py \
    --input_dir /path/to/pagexml_export \
    --output_dir ./data/my_dataset \
    --preserve-aspect-ratio \
    --target-height 128

# Step 2: Convert CSV β†’ CRNN-CTC format (required!)
python3 convert_to_pylaia.py \
    --input_csv ./data/my_dataset/train.csv \
    --output_dir ./data/crnn_train

python3 convert_to_pylaia.py \
    --input_csv ./data/my_dataset/val.csv \
    --output_dir ./data/crnn_val

# Step 3: Train CRNN-CTC model
python3 train_pylaia.py \
    --train_dir ./data/crnn_train \
    --val_dir ./data/crnn_val \
    --output_dir ./models/my_model \
    --batch_size 32 \
    --epochs 250

🌐 Web Interface

The web interface is the main way to use Polyscriptor β€” run inference locally or on a remote server and interact from any browser. No X11 forwarding needed; when running on a remote server, no local Python install is required either.

Starting the server

source htr_env/bin/activate    # Windows: htr_env\Scripts\activate
uvicorn web.polyscriptor_server:app --host 0.0.0.0 --port 8765   # run from the project root
# open http://localhost:8765

Works without a GPU. Commercial APIs (Gemini, Claude, OpenAI) and TrOCR run on CPU. CRNN-CTC also runs on CPU β€” inference is slower (~1–2 min/page) but fully functional, and our published Church Slavonic, Ukrainian and Glagolitic models all work this way. Only Qwen3-VL and LightOnOCR require a GPU.

Remote Access via SSH Tunnel

# On your laptop β€” tunnel port 8765 through SSH
ssh -L 8765:localhost:8765 user@your.server.edu

# Then open: http://localhost:8765
# No firewall issues β€” works on any university network

Features

  • Engine selection with dynamic configuration forms (CRNN-CTC, TrOCR, Kraken, etc.)
  • Image upload β€” drag-and-drop, file picker, or PDF upload (multi-page PDFs become batch items)
  • Segmentation β€” Kraken (neural blla or classical) with color-coded region overlay
  • Live transcription β€” Server-Sent Events stream, lines appear as processed
  • Batch queue β€” multi-image queue, drag-to-reorder, cancel, prev/next navigation
  • Inline editing β€” double-click any transcription line to correct it
  • Confidence filter β€” slider to dim low-confidence lines
  • Export β€” TXT, CSV, PAGE XML (single image or ZIP for entire batch)
  • Font selector β€” Monomakh Unicode (recommended for Church Slavonic), Old Standard TT, and others
  • Kraken model presets β€” local blla segmentation model and CATMuS-Print (Zenodo, one-click download); any other Kraken .mlmodel can be loaded from a local path
  • Resizable panels β€” drag handles to adjust column widths, saved across sessions

Running Tests

source htr_env/bin/activate
pip install pytest httpx
pytest web/tests/test_server.py -v

πŸ€– Pre-trained Models

Polyscriptor works with models from several sources. The table below lists tested, ready-to-use options for each engine.

CRNN-CTC

Ready-to-use models for Cyrillic and Glagolitic manuscripts, trained with Polyscriptor, are published on HuggingFace:

https://huggingface.co/achimrabus

Model Script / Language CER Description
crnn-ctc-church-slavonic Church Slavonic 2.89% QuantiSlav corpus (Elena Renje)
crnn-ctc-prosta-mova Prosta Mova (Old Ruthenian) 3.77% Continslav corpus (Martin Meindl)
crnn-ctc-ukrainian Ukrainian (Cyrillic) 4.76% 19th–20th c. manuscripts (MultiHTR)
crnn-ctc-glagolitic Croatian Glagolitic 5.33% Outputs Latin transliteration
crnn-ctc-russian Russian (Cyrillic) 6.92% 18th–20th c. manuscripts, print and typescript; pre- and post-reform orthography

All models run on CPU (no GPU required). They are presets of the CRNN-CTC engine: in a fresh clone they appear as "[Hugging Face, auto-download]" and are downloaded on first load (then cached); no models/ folder needs to be set up. A model placed under the registry path in models/ is used instead of the download.

TrOCR

A collection of TrOCR models for Cyrillic handwriting (Russian, Ukrainian, Church Slavonic) is maintained here:

https://huggingface.co/cyrillic-trocr

Built-in presets of the TrOCR engine (downloaded on first load). CER measured on 8 October 2026 with the decoding Polyscriptor uses (beam 4, no repeat block, max_length 128) on held-out validation pages:

Model Script / Language Validation CER Note
cyrillic-trocr/trocr-church-slavonic-handwritten Church Slavonic 3.80% 20,679 lines; same data as the CRNN-CTC model
cyrillic-trocr/trocr-ukrainian-handwritten Ukrainian 4.99% v2.0; 970 lines, 19th–20th c. manuscripts
achimrabus/trocr-glagolitic-handwritten Croatian Glagolitic 5.72% 1,551 lines; outputs Latin transliteration

Any other TrOCR model can be loaded by entering its HuggingFace ID or a local path.

Decoding. Polyscriptor decodes TrOCR without the n-gram repeat block (no_repeat_ngram_size=0) and with max_length=128. TrOCR's tokenizer splits Cyrillic into single letters or smaller pieces, so a repeat block suppresses ordinary letter sequences; switching it off improved every Cyrillic and Glagolitic model we tested. The block can be set as "Repeat block" in the web interface and the desktop GUI, or with --no-repeat-ngram-size; for Latin-script models it is untested.

LightOnOCR

Lightweight VLM models (~4 GB VRAM) for line-level HTR, compatible with the LightOnOCR engine:

Model Script / Language Description
wjbmattingly/LightOnOCR-2-1B-old-church-slavonic-line Church Slavonic Fine-tuned on QuantiSlav + Polyscriptor training data

Load via HuggingFace model ID in the LightOnOCR engine.

Qwen3-VL

Vision-language models fine-tuned for historical Slavic manuscripts (Church Slavonic, Glagolitic):

https://huggingface.co/wjbmattingly

Load any compatible Qwen3-VL model via its HuggingFace ID in the Qwen3-VL engine.

Kraken

The default blla model for neural line segmentation ships with Kraken itself; the repository carries an identical copy as pagexml/blla.mlmodel. Further segmentation and recognition models are published on Zenodo, for example:

Load a downloaded .mlmodel in the Kraken engine via its local path.

Party

Party OCR models with Church Slavonic in the pretraining data:

https://zenodo.org/records/15075344


πŸ“ Repository Structure

.
β”œβ”€β”€ train_pylaia.py                  # CRNN-CTC training script
β”œβ”€β”€ inference_pylaia_native.py       # CRNN-CTC inference (native Linux)
β”œβ”€β”€ inference_page.py                # Line segmentation + OCR pipeline
β”œβ”€β”€ transcription_gui_plugin.py      # Desktop GUI (PyQt6)
β”œβ”€β”€ polyscriptor_batch_gui.py        # Desktop batch GUI (PyQt6)
β”œβ”€β”€ batch_processing.py              # Batch processing CLI
β”œβ”€β”€ htr_engine_base.py              # HTR engine interface
β”‚
β”œβ”€β”€ engines/                         # HTR engine plugins
β”‚   β”œβ”€β”€ trocr_engine.py             # TrOCR transformer
β”‚   β”œβ”€β”€ pylaia_engine.py            # CRNN-CTC (Puigcerver CRNN)
β”‚   β”œβ”€β”€ qwen3_engine.py             # Qwen3-VL (local)
β”‚   β”œβ”€β”€ lighton_ocr_engine.py       # LightOnOCR VLM (lightweight)
β”‚   β”œβ”€β”€ churro_engine.py            # Churro (Qwen fork)
β”‚   β”œβ”€β”€ party_engine.py             # Party multilingual HTR
β”‚   β”œβ”€β”€ kraken_engine.py            # Kraken segmentation
β”‚   β”œβ”€β”€ commercial_api_engine.py    # Google Gemini, OpenAI GPT & Anthropic Claude APIs
β”‚   β”œβ”€β”€ openwebui_engine.py         # OpenWebUI local LLMs
β”‚   β”œβ”€β”€ paddle_engine.py            # PaddleOCR (subprocess, isolated venv)
β”‚   └── paddle_worker.py            # PaddleOCR worker (runs inside venv_paddle)
β”‚
β”œβ”€β”€ optimized_training.py            # TrOCR fine-tuning script
β”œβ”€β”€ transkribus_parser.py            # PAGE XML data preparation
β”œβ”€β”€ alto_parser.py                   # ALTO XML data preparation
β”œβ”€β”€ page_xml_exporter.py             # Export results to PAGE XML
β”œβ”€β”€ qwen3_prompts.py                 # Custom prompts for Qwen3-VL
β”‚
β”œβ”€β”€ requirements.txt                 # Python dependencies
β”‚
β”œβ”€β”€ web/                             # Browser-based web interface
β”‚   β”œβ”€β”€ polyscriptor_server.py      # FastAPI backend (SSE streaming)
β”‚   β”œβ”€β”€ static/
β”‚   β”‚   β”œβ”€β”€ index.html              # Single-page app
β”‚   β”‚   β”œβ”€β”€ app.js                  # State management, event bus
β”‚   β”‚   β”œβ”€β”€ app.css                 # Styles
β”‚   β”‚   └── components/             # ES6 modules (engine, viewer, transcription, batch)
β”‚   └── tests/
β”‚       └── test_server.py          # API tests (pytest + FastAPI TestClient)
β”‚
└── models/                          # Trained models (excluded from git)
    β”œβ”€β”€ pylaia_*/                    # CRNN-CTC model checkpoints
    └── trocr_*/                     # TrOCR fine-tuned models

πŸŽ“ Typical Workflow

Training a CRNN-CTC Model

  1. Export data from Transkribus (PAGE XML format)
  2. Parse with preprocessing:
    python3 transkribus_parser.py \
        --input_dir ./transkribus_export \
        --output_dir ./data/my_dataset \
        --preserve-aspect-ratio \
        --target-height 128 \
        # add --flip-rtl for Ottoman/Arabic/Hebrew manuscripts
  3. Convert to CRNN-CTC format:
    python3 convert_to_pylaia.py \
        --input_csv ./data/my_dataset/train.csv \
        --output_dir ./data/crnn_train
    python3 convert_to_pylaia.py \
        --input_csv ./data/my_dataset/val.csv \
        --output_dir ./data/crnn_val
  4. Train model:
    python3 train_pylaia.py \
        --train_dir ./data/crnn_train \
        --val_dir ./data/crnn_val \
        --output_dir ./models/my_model \
        --batch_size 32 \
        --epochs 250
  5. Use it: Model will appear in the CRNN-CTC engine dropdown (web interface and desktop GUI)

Using Trained Models

Trained models can be loaded in the web interface or the desktop GUI:

  • CRNN-CTC models: Select from dropdown or browse to model directory
  • TrOCR models: Specify HuggingFace Hub ID or local checkpoint path
  • Commercial APIs: Enter API keys in engine configuration

πŸ› οΈ Command-Line Inference

CRNN-CTC (Single Line)

python3 inference_pylaia_native.py \
    --checkpoint models/my_model/best_model.pt \
    --syms models/my_model/symbols.txt \
    --image line_image.png

CRNN-CTC (Full Page with Segmentation)

python3 inference_page.py \
    --image page.jpg \
    --checkpoint models/my_model/best_model.pt \
    --num-beams 4

πŸ“¦ Batch Processing

Batch Processing GUI

For processing multiple images or folders, use the batch processing GUI:

python3 polyscriptor_batch_gui.py

Features:

  • Process entire folders of images
  • Automatic PAGE XML detection (uses existing segmentation if available)
  • Progress tracking with live output
  • Export results to TXT, CSV, or PAGE XML
  • Resume interrupted processing

Batch Processing CLI

For scripted/automated workflows:

python3 batch_processing.py \
    --input-folder ./images \
    --engine crnn-ctc \
    --model-path models/my_model/best_model.pt \
    --segmentation-method kraken \
    --output-folder ./output \
    --use-pagexml

Key options:

  • --engine: crnn-ctc, TrOCR, Qwen3-VL, LightOnOCR, Party, Kraken, PaddleOCR
  • --segmentation-method: kraken (recommended), hpp (fast), none (pre-segmented)
  • --use-pagexml: Auto-detect and use existing PAGE XML segmentation
  • --resume: Skip already-processed files
  • --dry-run: Test without writing output
  • --flip-rtl: Flip line images horizontally for RTL scripts (see below)

↔️ RTL Manuscript Support (Ottoman, Arabic, Hebrew)

Some manuscript traditions are written right-to-left (RTL), but are transcribed with a Latin or Cyrillic alphabet that is read left-to-right (LTR). A key example is Ottoman Turkish manuscripts: the image is scanned RTL, but the ground truth transcription is a Latin-alphabet transliteration read LTR.

The problem: CRNN-CTC uses CTC alignment, which maps spatial position left-to-right to sequence position. If the image is RTL but the label is LTR, training fails. TrOCR (attention decoder) has the same issue. The fix is simple: flip all line images horizontally before training and during inference. This makes the visual input LTR, matching the LTR transcription.

What to do

Step 1 β€” Data preparation (flip images when parsing Transkribus export):

python3 transkribus_parser.py \
    --input_dir ./transkribus_export \
    --output_dir ./data/ottoman_dataset \
    --preserve-aspect-ratio \
    --target-height 128 \
    --flip-rtl          # <-- flip all line images horizontally

This saves flipped images to disk. The dataset_info.json will record "flip_rtl": true.

Step 2 β€” Training: No change needed. Train normally on the flipped images:

# CRNN-CTC
python3 train_pylaia.py \
    --train_dir ./data/crnn_train \
    --val_dir ./data/crnn_val \
    --output_dir ./models/ottoman_model \
    --epochs 250

# TrOCR
python3 optimized_training.py --config config_ottoman.yaml

Step 3 β€” Inference: You must use --flip-rtl at inference time too, because the model was trained on flipped images and expects the same orientation.

# Batch processing (CRNN-CTC or TrOCR)
python3 batch_processing.py \
    --input-folder HTR_Images/ottoman \
    --engine crnn-ctc \
    --model-path models/ottoman_model/best_model.pt \
    --flip-rtl

# Single-page inference (TrOCR)
python3 inference_page.py \
    --image page.jpg \
    --checkpoint models/ottoman_model/checkpoint-3000 \
    --flip-rtl

Web interface: Enable "RTL manuscript (flip line images)" in the CRNN-CTC or TrOCR config form. For TrOCR, this setting is applied at model load time (same as "Normalize Background") β€” reload the model after changing it.

Desktop GUI (PyQt): Enable the "RTL manuscript (flip line images)" checkbox in the engine's settings panel before loading the model.

Which engines support --flip-rtl?

Engine Supported Notes
CRNN-CTC Yes Flip applied per inference call β€” works as a runtime toggle
TrOCR Yes Flip applied at model load time β€” must reload if changed
API engines (Gemini, Claude, GPT) Not needed Vision models handle image orientation internally
Qwen3-VL / LightOnOCR Not needed Same β€” full-image vision models

Kraken Neural (blla) β€” reading order (text_direction)

When using Kraken Neural segmentation on RTL manuscripts, set the reading direction so that columns are returned in the correct order (right-most column first):

  • Web UI: "Reading direction" dropdown in Segmentation β†’ Kraken Neural options β†’ select RTL (Arabic, Ottoman, Hebrew, …)
  • PyQt GUI: "Direction" combo box in the blla segmentation settings
  • CLI: not yet exposed as a flag; defaults to horizontal-lr

This controls column ordering only β€” which column is listed first. It is independent of --flip-rtl, which flips the pixel content of each line. For Ottoman manuscripts you typically want both.


πŸ–¨οΈ PaddleOCR Engine

PaddleOCR performs its own text detection + recognition on whole pages β€” no pre-segmented lines needed. It runs in an isolated venv_paddle to avoid OpenCV conflicts with the main environment.

Setup

python3 -m venv venv_paddle
source venv_paddle/bin/activate
# CPU only:
pip install paddlepaddle paddleocr
# GPU (CUDA 12.x):
pip install paddleocr
pip install paddlepaddle-gpu==3.0.0 -f https://www.paddlepaddle.org.cn/packages/stable/cu126/
deactivate

Language codes

PaddleOCR uses ISO language codes (not script names). Enter the code in the "Language code" field of the engine config. Common examples:

Code Language / Script
en English
ch Chinese + English (strongest general model)
de or german German
fr or french French
ru Russian (Cyrillic)
uk Ukrainian (Cyrillic)
bg Bulgarian (Cyrillic)
la Latin (classical)
ar Arabic
japan Japanese
korean Korean

Note: Models download automatically on first use (~50–200 MB per script group). Only en is fetched during initial setup. Other language models cache in ~/.paddlex/official_models/. Full language list: https://paddlepaddle.github.io/PaddleOCR/main/en/ppocr/blog/multi_languages.html


πŸ–₯️ Remote Server Usage

Running on a remote Linux server without a display? You have several options.

Option 0: Web interface via SSH tunnel (recommended)

Best for: Interactive work, model comparison, correcting transcriptions; nothing to install locally. See Web Interface: start the server, then ssh -L 8765:localhost:8765 user@server and open http://localhost:8765.

Option 1: CLI Batch Processing

Best for: Production workflows, processing many images

# Process entire folders efficiently
python3 batch_processing.py \
    --input-folder HTR_Images/manuscripts \
    --engine crnn-ctc \
    --model-path models/crnn_ctc_model/best_model.pt \
    --use-pagexml \
    --output-folder output

Benefits: faster than GUI methods, no display overhead, scriptable

Option 2: X11 Forwarding (Interactive Work)

Best for: Interactive GUI work, visual parameter tuning, model comparison

Using MobaXterm on Windows:

  1. Install MobaXterm (X server auto-starts)
  2. SSH with X11 forwarding enabled
  3. Test: xclock & (should show clock window)
  4. Launch GUI: python3 transcription_gui_plugin.py

Performance: Good over LAN/local WiFi, slower over internet connections. Enable compression for best results.

Option 3: VNC (Alternative for Slow Connections)

Best for: When X11 is too slow (poor internet), extended GUI sessions, session persistence

# On server
vncserver :1 -geometry 1920x1080

# Connect from Windows using VNC viewer to: server:5901

Benefits: Better compression than X11, survives disconnects, works well over internet

Comparison

Method Speed Best For Network Type
CLI Batch Processing ⚑⚑⚑ Production, automation Any
Web UI ⚑⚑⚑ Interactive work, no install needed Any (SSH tunnel)
X11 Forwarding ⚑⚑ Interactive GUI work LAN/Local WiFi
X11 Forwarding ⚑ Light use only Internet
VNC/NoMachine ⚑⚑ Extended sessions, poor connections Any

βš™οΈ Configuration

CRNN-CTC Training Parameters

Key hyperparameters for optimal performance:

{
    "img_height": 128,           # Target image height
    "batch_size": 32,            # GPU-optimized (44GB VRAM)
    "num_epochs": 250,           # With early stopping
    "learning_rate": 0.0003,
    "early_stopping_patience": 15,
    "augment_train": True,       # Data augmentation
    "device": "cuda:0"
}

TrOCR Training Configuration

model_name: "kazars24/trocr-base-handwritten-ru"
data_root: "./processed_data"
batch_size: 16
epochs: 10
cache_images: true             # 10-50x faster data loading
fp16: true                     # Mixed precision training

🀝 Contributing

Contributions welcome! Areas of interest:

  1. New HTR engines: Add plugins for other HTR systems
  2. Model training: Share trained models for new scripts/languages
  3. Bug fixes: Especially inference/GUI issues
  4. Documentation: Improve guides and examples

πŸ“ License

MIT License

Copyright (c) 2025 Achim Rabus

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.


πŸ™ Acknowledgments


πŸ“§ Contact

For questions, bug reports, or collaboration inquiries:


πŸ”¬ Technical Notes

Critical Preprocessing for CRNN-CTC

Aspect Ratio Preservation is CRITICAL for high aspect ratio line images:

# ALWAYS use --preserve-aspect-ratio for manuscript lines
python3 transkribus_parser.py \
    --preserve-aspect-ratio \
    --target-height 128 \
    # ...other args

Without this, TrOCR's ViT encoder brutally resizes to 384Γ—384, causing 10.6x width compression for Ukrainian lines (4077Γ—357 β†’ 384Γ—384). Characters shrink from ~80px to ~7px width, making recognition nearly impossible.

Known Bugs (Fixed)

  1. KALDI Format Vocabulary: Train/inference scripts now auto-detect format
  2. <space> vs <SPACE>: Both cases handled correctly
  3. Vocabulary File Mismatch: Training scripts auto-copy vocabulary to model directory

About

Multi-engine ATR for multiple languages and scripts

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages