get_textpage_ocr(full=False) OCRs vector graphics, producing garbage words

[tenghian]

Description of the bug

Since the 1.27.2 change ("partial OCR now also OCRs vector graphics, not just images"), full=False runs Tesseract on any region MuPDF classifies as "no legible text"

ar2024_unknow_unicode.pdf

How to reproduce the bug

`
tp = page.get_textpage()

    if _UNKNOWN_UNICODE in page.get_text("text", textpage=tp):
        tp_ocr = page.get_textpage_ocr(flags=3, language='eng', dpi=300, full=False, tessdata=tessdata)
        word_blocks = page.get_text("words", flags=flags, textpage=tp_ocr)

`

PyMuPDF version

1.28.0

Operating system

Windows

Python version

3.12

[JorjMcKie]

We have indeed changed our approach here: away from iterating over images to looking at the page as a whole. Previously we didn't OCR � when using full=False. Neither did we take into account vectors doing text mimicry.

For the most part, your problem is caused by Tesseract's over-eagerness: It indeed has a low threshold in assuming that something is worth OCR-ing. It regards general vectors like lines or rectangles much to often. We have no influence here and we also do not support alternative OCR engines in this context.

The second problem: You are trying to detect OCR need, but your check is incomplete.
The first page contains only one � out of 676 ok characters. You should consider introducing a more sophisticated check, maybe a percentage-based approach.
The second page is much different: 573 out of 742 are illegible.

I recommend considering the following:

1. Extract text conventionally via get_text("dict"). Execute Tesseract OCR for the bbox of each span that contains a �. Look at this recipe for how to approach this.
2. Use the page analyzer of pymupdf4llm to see if a page needs OCR. It uses the following logic:

import pymupdf
import pymupdf4llm
from pprint import pp
analyze = pymupdf4llm.ocr.analyze_page.analyze_page

doc = pymupdf.open("ar2024_unknow_unicode.pdf")
page = doc[0]

pp(analyze(page))
{'covered': Rect(56.858001708984375, 18.81598663330078, 569.9749755859375, 657.7340698242188),
 'img_joins': 0.23674984093456133,
 'img_area': 0.04145623438648762,
 'txt_joins': 0.8489164265522499,
 'txt_area': 0.08898051574209948,
 'vec_joins': 0.9117148291676324,
 'vec_area': 0.6797198931944479,
 'chars_total': 676,
 'chars_bad': 1,  # 0xFFFD count
 'bad_areas': 0.00015417165507585623,
 'ocr_spans': 0,
 'pixmap': None,
 'vec_norects': 77,  # number of vectors that MIGHT mimic characters
 'needs_ocr': False,  # OCR not recommended
 'reason': None,
 'probability': 0.8193221092224121}  # probability below threshold 93%

So this function does a detailed analysis and comes to the conclusion that OCR is not needed.
The result for the second page clearly recommends OCR and gives us the reason why:

{'covered': Rect(22.25029945373535, 18.81598663330078, 541.2203979492188, 678.5220336914062),
 'img_joins': 0.5118280247708684,
 'img_area': 0.4762521215056628,
 'txt_joins': 0.3988905456292418,
 'txt_area': 0.09699813982510444,  # text area is 9.6% of "covered" area
 'vec_joins': 0.3379046622108768,
 'vec_area': 0.028636319539308345,
 'chars_total': 742,
 'chars_bad': 573,
 'bad_areas': 0.07193382842621195,  # 7% of "covered" area
 'ocr_spans': 0,
 'pixmap': None,
 'vec_norects': 1,
 'needs_ocr': True,  # recommend OCR
 'reason': 'chars_bad',  # for this reason 
 'probability': None}