Repository navigation
A lot of memory used on newspaper pages? #110
Description
In OCR-D/quiver-benchmarks#22, @stweil mentions 118 GB being used for newspaper pages.
- Reproduce
- Can we test for this somehow
{{ message }}
In OCR-D/quiver-benchmarks#22, @stweil mentions 118 GB being used for newspaper pages.
Activity
mikegerber commentedon Feb 22, 2024
Might have overlooked this because a. our servers have a lot of memory and b. I didn't process a lot of newspapers.
I asked @stweil for the input data. Need to check if I have some newspaper pages readily segmented.
Options used seem to be
-I OCR-D-SEG-LINE-RESEG-DEWARP -O OCR-D-OCR -P checkpoint_dir qurator-gt4histocr-1.0"[-]A lot of memory used on newspaper pages[/-][+]A lot of memory used on newspaper pages?[/+]mikegerber commentedon Feb 28, 2024
I don't have the data for the issue mentioned in OCR-D/quiver-benchmarks#22, tried to produce something similar but failed due to an unrelated issue.
→ Trying with some other data supplied by @cneud
mikegerber commentedon Feb 28, 2024
Yeah, ran into another unrelated issue first: OCR-D/core#1179
mikegerber commentedon Feb 28, 2024
The page I used only had 365 lines, didn't see anything more than 1.8 GB RSS ("not great, not terrible").
There is something else wrong, though, it seems to use the raw (RGB) images for some lines, this does not make sense. But the XML may be not be 100% as I imported it etc. pp.
mikegerber commentedon Mar 5, 2024
The workspace also showed signs of OCR-D/core#1195, so I'll try again first, with METS caching disabled.
mikegerber commentedon Mar 5, 2024
I've redone the segmentation, no "raw image" problem anymore. Probably just because I couldn't figure out how to fix up the XML so i works properly with the AlternativeImage logic