[proposal] Reduce memory and event-loop cost of the Tesseract fallback preprocessing

[Ishan-Karpe]

Hi, I'm Ishan, I'm a high school junior working on Maxun through CodeDay Labs.

I've been reading through DocumentInterpreter.ts and wanted to check something before assuming.

preprocessPageImage() (the Sauvola binarization) seems to only be reachable from
the Tesseract paths. ocrPDF calls it on every rasterized page but paddleOCRPDF
doesn't, and PaddleOCR is the one that runs first, with Tesseract only firing on
a throw. So unless I'm misreading it, the binarization doesn't actually run in
the common case.

Is that on purpose? I could see it being deliberate, since Paddle does its own
normalization and binarizing first might hurt its detection model. If it isn't,
I'd like to look into it, though I'd want to build some kind of OCR quality
benchmark first, since there's no way to measure whether preprocessing helps or
hurts right now.

[amhsirak]

@Ishan-Karpe Hello! No, you're not misreading it. preprocessPageImage() only ever gets called from the Tesseract path, and since Paddle runs first with Tesseract only kicking in when Paddle throws, that binarization basically never runs in practice. It is intentional though, and your guess about why is spot on. Paddle handles its own preprocessing, so binarizing beforehand would more likely hurt than help, whereas for Tesseract it's a clear upgrade. That said, you did find something real. The function name makes it look like shared pipeline code when it's very much a Tesseract thing, which is exactly why it tripped you up. Tesseract path being rarely hit is on purpose. It's the fallback for when Paddle falls over, so it's not dead code, and we do want it kept in good shape.

[Ishan-Karpe]

Thanks @amhsirak, that clears it up.

The line that stuck with me was "it's not dead code, and we do want it kept in good shape" because while I was tracing that path I ran into two things that push against it.

preprocessPageImage() allocates a few full-page buffers for every page it handles: a greyscale copy, two lookup tables for the windowed math, and the raw pixel data. On an A4 page at the scale the PDF path renders at, about 8 million pixels, that comes to
roughly 192MB. That is arithmetic off the array sizes rather than something I have measured, and the real figure would be higher once canvas and decode overhead are counted. Most of it is the two lookup tables. The running-sum one could be a narrower integer type, though the sum-of-squares needs the range at full-page size, so the real lever is probably working through the page in tiles instead, which bounds both tables and avoids holding either at full size.

The second is that the thresholding loop runs straight through with no yield, so it holds the event loop for the whole pass. Once per page means a multi-page scan that hits the fallback stalls things repeatedly rather than just once.

If you would want a PR for this, I would measure it properly first so those numbers stop being estimates and there is a baseline to show the change actually helped, then bring the allocation down and get the loop off the main thread, and check the output image comes out
byte-identical before and after. Fallback path only, nothing touching Paddle or OCR results.

One thing I would want your call on: there is no test setup in the repo right now, so for the before and after measurement I would default to a standalone script under server/src/ that CI never runs, unless you would rather I bring in a framework.

[Ishan-Karpe]

@amhsirak just a quick follow up for the proposal