A comprehensive toolkit for training and comparing different Handwritten Text Recognition (HTR) engines on historical manuscript datasets. Supports TrOCR, CRNN-CTC, Qwen3-VL, LightOnOCR, Party, and Kraken engines through a browser-based web interface (plus a PyQt6 desktop GUI and a batch CLI).
Primary Focus: Cyrillic manuscripts (Russian, Ukrainian, Church Slavonic, Glagolitic)
- TrOCR: Transformer-based OCR (line-level)
- CRNN-CTC: Puigcerver CRNN with CTC decoding (line-level, PyLaia-inspired)
- Qwen3-VL: Vision-Language Model (line/page-level, custom prompts)
- LightOnOCR: Lightweight VLM (~4GB VRAM, line-level, fine-tuned variants)
- Churro: Qwen fork, experimental (line/page-level, custom prompts)
- Party: Transformer-based HTR (line-level, multilingual)
- Kraken: Segmentation & recognition
- PaddleOCR: Multi-language printed/mixed text detection + recognition (whole-page, subprocess-isolated)
- Commercial APIs: Google Gemini, Anthropic Claude Vision (via API keys)
- Local LLMs: OpenWebUI integration for local vision models
- Unified interface: All models accessible through same engine plugin system
- Web interface: Upload, segment, transcribe, correct and export in the browser; runs locally or on a server
- Engine comparison: Compare engines side-by-side (web interface and desktop GUI)
- Model management: Easy switching between trained models and API providers
- Export formats: TXT, CSV, PAGE XML
- CRNN-CTC: Custom CRNN training with PAGE XML support
- TrOCR: Fine-tuning pipeline with image caching (10-50x faster)
- Data preparation: Transkribus PAGE XML parser
- Line segmentation (automatic or PAGE XML-based)
- Custom prompt support (Qwen3-VL)
- Batch processing
- PAGE XML import/export
# Clone repository
git clone https://github.com/achimrabus/polyscriptor.git
cd polyscriptor
# Create virtual environment
# Requires Python 3.10β3.12 (Python 3.13+ may lack prebuilt wheels for some
# packages; check your version with: python --version)
python3 -m venv htr_env
source htr_env/bin/activate # Linux/Mac
# or: htr_env\Scripts\activate # WindowsInstall everything in one pip call into the fresh environment. requirements-kraken.txt
adds Kraken segmentation; it is optional but recommended (without it, the fast but much weaker
HPP segmentation is used).
GPU install (CUDA 12.8 β Linux/Windows with NVIDIA GPU):
pip install -r requirements-gpu.txt -r requirements.txt -r requirements-kraken.txt \
--extra-index-url https://download.pytorch.org/whl/cu128CPU-only install (no GPU required):
# Linux/Mac:
pip install -r requirements.txt -r requirements-kraken.txt
# Windows (avoids a torch DLL load error on CPU-only machines):
pip install -r requirements.txt -r requirements-kraken.txt --extra-index-url https://download.pytorch.org/whl/cpuDo not add Kraken to an existing environment later (
pip install krakenorpip install -r requirements-kraken.txtas a second step). Kraken pins several shared packages tightly; installed afterwards, pip downgrades them andtransformersstops importing. If you need Kraken in an environment that was set up without it, create a new environment and use the one-call install above.
source htr_env/bin/activate # Windows: htr_env\Scripts\activate
uvicorn web.polyscriptor_server:app --host 0.0.0.0 --port 8765Open http://localhost:8765 in your browser. On a remote server, tunnel the port from your
laptop with ssh -L 8765:localhost:8765 user@server and open the same address locally. No
display, X11 or Qt is needed. Details: Web Interface.
Alternative: desktop GUI (PyQt6)
python3 transcription_gui_plugin.py # single pages
python3 polyscriptor_batch_gui.py # foldersLinux: PyQt6 (Qt β₯ 6.5) needs the system library
libxcb-cursor0. If the GUI fails with "Could not load the Qt platform plugin "xcb"", install it:sudo apt install libxcb-cursor0(Debian/Ubuntu).
For many pages: batch CLI
python3 batch_processing.py \
--input-folder HTR_Images/my_folder \
--engine crnn-ctc \
--model-path models/crnn_ctc_model/best_model.pt \
--segmentation-method kraken-blla \
--use-pagexml# Step 1: Parse PAGE XML export β CSV format
# Works with Transkribus exports (namespace 2013-07-15) and
# eScriptorium/Kraken exports (namespace 2019-07-15) β both are auto-detected.
python3 transkribus_parser.py \
--input_dir /path/to/pagexml_export \
--output_dir ./data/my_dataset \
--preserve-aspect-ratio \
--target-height 128
# Step 2: Convert CSV β CRNN-CTC format (required!)
python3 convert_to_pylaia.py \
--input_csv ./data/my_dataset/train.csv \
--output_dir ./data/crnn_train
python3 convert_to_pylaia.py \
--input_csv ./data/my_dataset/val.csv \
--output_dir ./data/crnn_val
# Step 3: Train CRNN-CTC model
python3 train_pylaia.py \
--train_dir ./data/crnn_train \
--val_dir ./data/crnn_val \
--output_dir ./models/my_model \
--batch_size 32 \
--epochs 250The web interface is the main way to use Polyscriptor β run inference locally or on a remote server and interact from any browser. No X11 forwarding needed; when running on a remote server, no local Python install is required either.
source htr_env/bin/activate # Windows: htr_env\Scripts\activate
uvicorn web.polyscriptor_server:app --host 0.0.0.0 --port 8765 # run from the project root
# open http://localhost:8765Works without a GPU. Commercial APIs (Gemini, Claude, OpenAI) and TrOCR run on CPU. CRNN-CTC also runs on CPU β inference is slower (~1β2 min/page) but fully functional, and our published Church Slavonic, Ukrainian and Glagolitic models all work this way. Only Qwen3-VL and LightOnOCR require a GPU.
# On your laptop β tunnel port 8765 through SSH
ssh -L 8765:localhost:8765 user@your.server.edu
# Then open: http://localhost:8765
# No firewall issues β works on any university network- Engine selection with dynamic configuration forms (CRNN-CTC, TrOCR, Kraken, etc.)
- Image upload β drag-and-drop, file picker, or PDF upload (multi-page PDFs become batch items)
- Segmentation β Kraken (neural blla or classical) with color-coded region overlay
- Live transcription β Server-Sent Events stream, lines appear as processed
- Batch queue β multi-image queue, drag-to-reorder, cancel, prev/next navigation
- Inline editing β double-click any transcription line to correct it
- Confidence filter β slider to dim low-confidence lines
- Export β TXT, CSV, PAGE XML (single image or ZIP for entire batch)
- Font selector β Monomakh Unicode (recommended for Church Slavonic), Old Standard TT, and others
- Kraken model presets β local blla segmentation model and CATMuS-Print (Zenodo, one-click download); any other Kraken
.mlmodelcan be loaded from a local path - Resizable panels β drag handles to adjust column widths, saved across sessions
source htr_env/bin/activate
pip install pytest httpx
pytest web/tests/test_server.py -vPolyscriptor works with models from several sources. The table below lists tested, ready-to-use options for each engine.
Ready-to-use models for Cyrillic and Glagolitic manuscripts, trained with Polyscriptor, are published on HuggingFace:
https://huggingface.co/achimrabus
| Model | Script / Language | CER | Description |
|---|---|---|---|
| crnn-ctc-church-slavonic | Church Slavonic | 2.89% | QuantiSlav corpus (Elena Renje) |
| crnn-ctc-prosta-mova | Prosta Mova (Old Ruthenian) | 3.77% | Continslav corpus (Martin Meindl) |
| crnn-ctc-ukrainian | Ukrainian (Cyrillic) | 4.76% | 19thβ20th c. manuscripts (MultiHTR) |
| crnn-ctc-glagolitic | Croatian Glagolitic | 5.33% | Outputs Latin transliteration |
| crnn-ctc-russian | Russian (Cyrillic) | 6.92% | 18thβ20th c. manuscripts, print and typescript; pre- and post-reform orthography |
All models run on CPU (no GPU required). They are presets of the CRNN-CTC engine: in a fresh
clone they appear as "[Hugging Face, auto-download]" and are downloaded on first load (then
cached); no models/ folder needs to be set up. A model placed under the registry path in
models/ is used instead of the download.
A collection of TrOCR models for Cyrillic handwriting (Russian, Ukrainian, Church Slavonic) is maintained here:
https://huggingface.co/cyrillic-trocr
Built-in presets of the TrOCR engine (downloaded on first load). CER measured on 8 October 2026 with
the decoding Polyscriptor uses (beam 4, no repeat block, max_length 128) on held-out validation pages:
| Model | Script / Language | Validation CER | Note |
|---|---|---|---|
| cyrillic-trocr/trocr-church-slavonic-handwritten | Church Slavonic | 3.80% | 20,679 lines; same data as the CRNN-CTC model |
| cyrillic-trocr/trocr-ukrainian-handwritten | Ukrainian | 4.99% | v2.0; 970 lines, 19thβ20th c. manuscripts |
| achimrabus/trocr-glagolitic-handwritten | Croatian Glagolitic | 5.72% | 1,551 lines; outputs Latin transliteration |
Any other TrOCR model can be loaded by entering its HuggingFace ID or a local path.
Decoding. Polyscriptor decodes TrOCR without the n-gram repeat block (no_repeat_ngram_size=0)
and with max_length=128. TrOCR's tokenizer splits Cyrillic into single letters or smaller pieces,
so a repeat block suppresses ordinary letter sequences; switching it off improved every Cyrillic and
Glagolitic model we tested. The block can be set as "Repeat block" in the web interface and the
desktop GUI, or with --no-repeat-ngram-size; for Latin-script models it is untested.
Lightweight VLM models (~4 GB VRAM) for line-level HTR, compatible with the LightOnOCR engine:
| Model | Script / Language | Description |
|---|---|---|
| wjbmattingly/LightOnOCR-2-1B-old-church-slavonic-line | Church Slavonic | Fine-tuned on QuantiSlav + Polyscriptor training data |
Load via HuggingFace model ID in the LightOnOCR engine.
Vision-language models fine-tuned for historical Slavic manuscripts (Church Slavonic, Glagolitic):
https://huggingface.co/wjbmattingly
Load any compatible Qwen3-VL model via its HuggingFace ID in the Qwen3-VL engine.
The default blla model for neural line segmentation ships with Kraken itself; the repository
carries an identical copy as pagexml/blla.mlmodel. Further segmentation and recognition models
are published on Zenodo, for example:
- CATMuS-Print β printed text, multilingual (built-in preset)
- Generic HTR model for Old Cyrillic uncial and semi-uncial β Church Slavonic, 11thβ16th c.
Load a downloaded .mlmodel in the Kraken engine via its local path.
Party OCR models with Church Slavonic in the pretraining data:
https://zenodo.org/records/15075344
.
βββ train_pylaia.py # CRNN-CTC training script
βββ inference_pylaia_native.py # CRNN-CTC inference (native Linux)
βββ inference_page.py # Line segmentation + OCR pipeline
βββ transcription_gui_plugin.py # Desktop GUI (PyQt6)
βββ polyscriptor_batch_gui.py # Desktop batch GUI (PyQt6)
βββ batch_processing.py # Batch processing CLI
βββ htr_engine_base.py # HTR engine interface
β
βββ engines/ # HTR engine plugins
β βββ trocr_engine.py # TrOCR transformer
β βββ pylaia_engine.py # CRNN-CTC (Puigcerver CRNN)
β βββ qwen3_engine.py # Qwen3-VL (local)
β βββ lighton_ocr_engine.py # LightOnOCR VLM (lightweight)
β βββ churro_engine.py # Churro (Qwen fork)
β βββ party_engine.py # Party multilingual HTR
β βββ kraken_engine.py # Kraken segmentation
β βββ commercial_api_engine.py # Google Gemini, OpenAI GPT & Anthropic Claude APIs
β βββ openwebui_engine.py # OpenWebUI local LLMs
β βββ paddle_engine.py # PaddleOCR (subprocess, isolated venv)
β βββ paddle_worker.py # PaddleOCR worker (runs inside venv_paddle)
β
βββ optimized_training.py # TrOCR fine-tuning script
βββ transkribus_parser.py # PAGE XML data preparation
βββ alto_parser.py # ALTO XML data preparation
βββ page_xml_exporter.py # Export results to PAGE XML
βββ qwen3_prompts.py # Custom prompts for Qwen3-VL
β
βββ requirements.txt # Python dependencies
β
βββ web/ # Browser-based web interface
β βββ polyscriptor_server.py # FastAPI backend (SSE streaming)
β βββ static/
β β βββ index.html # Single-page app
β β βββ app.js # State management, event bus
β β βββ app.css # Styles
β β βββ components/ # ES6 modules (engine, viewer, transcription, batch)
β βββ tests/
β βββ test_server.py # API tests (pytest + FastAPI TestClient)
β
βββ models/ # Trained models (excluded from git)
βββ pylaia_*/ # CRNN-CTC model checkpoints
βββ trocr_*/ # TrOCR fine-tuned models
- Export data from Transkribus (PAGE XML format)
- Parse with preprocessing:
python3 transkribus_parser.py \ --input_dir ./transkribus_export \ --output_dir ./data/my_dataset \ --preserve-aspect-ratio \ --target-height 128 \ # add --flip-rtl for Ottoman/Arabic/Hebrew manuscripts - Convert to CRNN-CTC format:
python3 convert_to_pylaia.py \ --input_csv ./data/my_dataset/train.csv \ --output_dir ./data/crnn_train python3 convert_to_pylaia.py \ --input_csv ./data/my_dataset/val.csv \ --output_dir ./data/crnn_val - Train model:
python3 train_pylaia.py \ --train_dir ./data/crnn_train \ --val_dir ./data/crnn_val \ --output_dir ./models/my_model \ --batch_size 32 \ --epochs 250 - Use it: Model will appear in the CRNN-CTC engine dropdown (web interface and desktop GUI)
Trained models can be loaded in the web interface or the desktop GUI:
- CRNN-CTC models: Select from dropdown or browse to model directory
- TrOCR models: Specify HuggingFace Hub ID or local checkpoint path
- Commercial APIs: Enter API keys in engine configuration
python3 inference_pylaia_native.py \
--checkpoint models/my_model/best_model.pt \
--syms models/my_model/symbols.txt \
--image line_image.pngpython3 inference_page.py \
--image page.jpg \
--checkpoint models/my_model/best_model.pt \
--num-beams 4For processing multiple images or folders, use the batch processing GUI:
python3 polyscriptor_batch_gui.pyFeatures:
- Process entire folders of images
- Automatic PAGE XML detection (uses existing segmentation if available)
- Progress tracking with live output
- Export results to TXT, CSV, or PAGE XML
- Resume interrupted processing
For scripted/automated workflows:
python3 batch_processing.py \
--input-folder ./images \
--engine crnn-ctc \
--model-path models/my_model/best_model.pt \
--segmentation-method kraken \
--output-folder ./output \
--use-pagexmlKey options:
--engine: crnn-ctc, TrOCR, Qwen3-VL, LightOnOCR, Party, Kraken, PaddleOCR--segmentation-method: kraken (recommended), hpp (fast), none (pre-segmented)--use-pagexml: Auto-detect and use existing PAGE XML segmentation--resume: Skip already-processed files--dry-run: Test without writing output--flip-rtl: Flip line images horizontally for RTL scripts (see below)
Some manuscript traditions are written right-to-left (RTL), but are transcribed with a Latin or Cyrillic alphabet that is read left-to-right (LTR). A key example is Ottoman Turkish manuscripts: the image is scanned RTL, but the ground truth transcription is a Latin-alphabet transliteration read LTR.
The problem: CRNN-CTC uses CTC alignment, which maps spatial position left-to-right to sequence position. If the image is RTL but the label is LTR, training fails. TrOCR (attention decoder) has the same issue. The fix is simple: flip all line images horizontally before training and during inference. This makes the visual input LTR, matching the LTR transcription.
Step 1 β Data preparation (flip images when parsing Transkribus export):
python3 transkribus_parser.py \
--input_dir ./transkribus_export \
--output_dir ./data/ottoman_dataset \
--preserve-aspect-ratio \
--target-height 128 \
--flip-rtl # <-- flip all line images horizontallyThis saves flipped images to disk. The dataset_info.json will record "flip_rtl": true.
Step 2 β Training: No change needed. Train normally on the flipped images:
# CRNN-CTC
python3 train_pylaia.py \
--train_dir ./data/crnn_train \
--val_dir ./data/crnn_val \
--output_dir ./models/ottoman_model \
--epochs 250
# TrOCR
python3 optimized_training.py --config config_ottoman.yamlStep 3 β Inference: You must use --flip-rtl at inference time too, because the model was trained on flipped images and expects the same orientation.
# Batch processing (CRNN-CTC or TrOCR)
python3 batch_processing.py \
--input-folder HTR_Images/ottoman \
--engine crnn-ctc \
--model-path models/ottoman_model/best_model.pt \
--flip-rtl
# Single-page inference (TrOCR)
python3 inference_page.py \
--image page.jpg \
--checkpoint models/ottoman_model/checkpoint-3000 \
--flip-rtlWeb interface: Enable "RTL manuscript (flip line images)" in the CRNN-CTC or TrOCR config form. For TrOCR, this setting is applied at model load time (same as "Normalize Background") β reload the model after changing it.
Desktop GUI (PyQt): Enable the "RTL manuscript (flip line images)" checkbox in the engine's settings panel before loading the model.
| Engine | Supported | Notes |
|---|---|---|
| CRNN-CTC | Yes | Flip applied per inference call β works as a runtime toggle |
| TrOCR | Yes | Flip applied at model load time β must reload if changed |
| API engines (Gemini, Claude, GPT) | Not needed | Vision models handle image orientation internally |
| Qwen3-VL / LightOnOCR | Not needed | Same β full-image vision models |
When using Kraken Neural segmentation on RTL manuscripts, set the reading direction so that columns are returned in the correct order (right-most column first):
- Web UI: "Reading direction" dropdown in Segmentation β Kraken Neural options β select
RTL (Arabic, Ottoman, Hebrew, β¦) - PyQt GUI: "Direction" combo box in the blla segmentation settings
- CLI: not yet exposed as a flag; defaults to
horizontal-lr
This controls column ordering only β which column is listed first. It is independent of --flip-rtl, which flips the pixel content of each line. For Ottoman manuscripts you typically want both.
PaddleOCR performs its own text detection + recognition on whole pages β no pre-segmented lines needed. It runs in an isolated venv_paddle to avoid OpenCV conflicts with the main environment.
python3 -m venv venv_paddle
source venv_paddle/bin/activate
# CPU only:
pip install paddlepaddle paddleocr
# GPU (CUDA 12.x):
pip install paddleocr
pip install paddlepaddle-gpu==3.0.0 -f https://www.paddlepaddle.org.cn/packages/stable/cu126/
deactivatePaddleOCR uses ISO language codes (not script names). Enter the code in the "Language code" field of the engine config. Common examples:
| Code | Language / Script |
|---|---|
en |
English |
ch |
Chinese + English (strongest general model) |
de or german |
German |
fr or french |
French |
ru |
Russian (Cyrillic) |
uk |
Ukrainian (Cyrillic) |
bg |
Bulgarian (Cyrillic) |
la |
Latin (classical) |
ar |
Arabic |
japan |
Japanese |
korean |
Korean |
Note: Models download automatically on first use (~50β200 MB per script group). Only
enis fetched during initial setup. Other language models cache in~/.paddlex/official_models/. Full language list: https://paddlepaddle.github.io/PaddleOCR/main/en/ppocr/blog/multi_languages.html
Running on a remote Linux server without a display? You have several options.
Best for: Interactive work, model comparison, correcting transcriptions; nothing to install locally.
See Web Interface: start the server, then ssh -L 8765:localhost:8765 user@server and open http://localhost:8765.
Best for: Production workflows, processing many images
# Process entire folders efficiently
python3 batch_processing.py \
--input-folder HTR_Images/manuscripts \
--engine crnn-ctc \
--model-path models/crnn_ctc_model/best_model.pt \
--use-pagexml \
--output-folder outputBenefits: faster than GUI methods, no display overhead, scriptable
Best for: Interactive GUI work, visual parameter tuning, model comparison
Using MobaXterm on Windows:
- Install MobaXterm (X server auto-starts)
- SSH with X11 forwarding enabled
- Test:
xclock &(should show clock window) - Launch GUI:
python3 transcription_gui_plugin.py
Performance: Good over LAN/local WiFi, slower over internet connections. Enable compression for best results.
Best for: When X11 is too slow (poor internet), extended GUI sessions, session persistence
# On server
vncserver :1 -geometry 1920x1080
# Connect from Windows using VNC viewer to: server:5901Benefits: Better compression than X11, survives disconnects, works well over internet
| Method | Speed | Best For | Network Type |
|---|---|---|---|
| CLI Batch Processing | β‘β‘β‘ | Production, automation | Any |
| Web UI | β‘β‘β‘ | Interactive work, no install needed | Any (SSH tunnel) |
| X11 Forwarding | β‘β‘ | Interactive GUI work | LAN/Local WiFi |
| X11 Forwarding | β‘ | Light use only | Internet |
| VNC/NoMachine | β‘β‘ | Extended sessions, poor connections | Any |
Key hyperparameters for optimal performance:
{
"img_height": 128, # Target image height
"batch_size": 32, # GPU-optimized (44GB VRAM)
"num_epochs": 250, # With early stopping
"learning_rate": 0.0003,
"early_stopping_patience": 15,
"augment_train": True, # Data augmentation
"device": "cuda:0"
}model_name: "kazars24/trocr-base-handwritten-ru"
data_root: "./processed_data"
batch_size: 16
epochs: 10
cache_images: true # 10-50x faster data loading
fp16: true # Mixed precision trainingContributions welcome! Areas of interest:
- New HTR engines: Add plugins for other HTR systems
- Model training: Share trained models for new scripts/languages
- Bug fixes: Especially inference/GUI issues
- Documentation: Improve guides and examples
MIT License
Copyright (c) 2025 Achim Rabus
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
- PyLaia (architecture inspiration for CRNN-CTC engine): https://github.com/jpuigcerver/PyLaia
- TrOCR: Microsoft's Transformer-based OCR: https://huggingface.co/microsoft/trocr-base-handwritten
- LightOnOCR: Lightweight VLM for OCR: https://huggingface.co/lightonai/LightOnOCR-2-1B-base
- Party: PAge-wise Recognition of Text-y https://github.com/mittagessen/party/
- Transkribus: Transcription, training, and inference plattform: https://app.transkribus.org/
- Qwen3-VL: Alibaba's Vision-Language Model: https://github.com/QwenLM/Qwen3-VL
- William Mattingly: Support with VLM fine-tuning and Church Slavonic models: https://huggingface.co/wjbmattingly
For questions, bug reports, or collaboration inquiries:
- GitHub Issues: Create an issue
Aspect Ratio Preservation is CRITICAL for high aspect ratio line images:
# ALWAYS use --preserve-aspect-ratio for manuscript lines
python3 transkribus_parser.py \
--preserve-aspect-ratio \
--target-height 128 \
# ...other argsWithout this, TrOCR's ViT encoder brutally resizes to 384Γ384, causing 10.6x width compression for Ukrainian lines (4077Γ357 β 384Γ384). Characters shrink from ~80px to ~7px width, making recognition nearly impossible.
- KALDI Format Vocabulary: Train/inference scripts now auto-detect format
<space>vs<SPACE>: Both cases handled correctly- Vocabulary File Mismatch: Training scripts auto-copy vocabulary to model directory