fix: Form XObject text garbled when /Resources is an indirect reference - #407
Open
MADENIYOU wants to merge 1 commit into
Open
fix: Form XObject text garbled when /Resources is an indirect reference#407MADENIYOU wants to merge 1 commit into
MADENIYOU wants to merge 1 commit into
Conversation
walk_xobject_fonts resolved a Form XObject's /Resources entry with `stream.dict.get(b"Resources").and_then(Object::as_dict)` — handling only the inline-dictionary case. Per spec, /Resources (like most PDF dictionary-valued entries) may equally be an indirect reference to a dictionary. When it was, `.as_dict()` on the Reference object failed silently, and the Form's fonts (including any /ToUnicode CMap) were never collected — producing garbled or empty text and incorrectly routing an otherwise-text-based page to OCR. Added a shared resolve_dict() helper and used it everywhere a dictionary-or-reference value needs resolving in this function (/Resources, /Font, and the Font dict's own entries), consistent with how /XObject was already resolved a few lines above. Also fixed a related gap the issue flagged: the fast/page-selected CMap path (from_doc_pages_fast) skipped Form XObject font collection entirely (collect_cmaps_from_xobjects was only called when !skip_truetype_fallback), rather than still collecting cheap primary ToUnicode CMaps while only skipping the expensive embedded-font fallback parsing. Threaded skip_truetype_fallback through collect_cmaps_from_xobjects/walk_xobject_fonts so fast mode still discovers primary Form XObject CMaps. Removed collect_cmaps_from_fonts, a thin wrapper that became dead code once walk_xobject_fonts called collect_cmaps_from_fonts_inner directly to pass the propagated flag. Fixes firecrawl#398. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
There was a problem hiding this comment.
No issues found across 1 file
Shadow auto-approve: would auto-approve. Focused bug fix: resolves indirect /Resources dictionaries so Form fonts' ToUnicode CMaps are collected, and fast mode now still discovers primary CMaps while skipping only expensive fallback. Tests pin both paths; no rollout, contract, or ops change.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #398.
Root cause
FontCMaps::walk_xobject_fonts(src/tounicode.rs) resolved a Form XObject's/Resourcesentry with:This only handles
/Resourcesas an inline dictionary. Per spec, it may equally be an indirect reference to a dictionary — and when it is,.as_dict()on theObject::Referencefails, so theif letsilently doesn't run and the Form's fonts (including any/ToUnicodeCMap) are never collected. That can produce garbled/empty Markdown and incorrectly route an otherwise-text-based page to OCR.Interestingly,
/XObjectresolution a few lines above this already handled both shapes correctly (Ok(Object::Reference(id)) => ...,Ok(Object::Dictionary(dict)) => ...) —/Resourcesand/Fontjust hadn't been brought in line with that.Fix
Added a shared
resolve_dict()helper and used it for every dictionary-or-reference lookup in this function (/Resources,/Font, and the font dict's own entries), matching the pattern already used for/XObject.Also fixed a related gap: the fast/page-selected CMap path (
from_doc_pages_fast) skipped Form XObject font collection entirely (collect_cmaps_from_xobjectswas gated behind!skip_truetype_fallback), rather than still collecting cheap primary ToUnicode CMaps while only skipping the expensive embedded-font fallback parsing. Threadedskip_truetype_fallbackthroughcollect_cmaps_from_xobjects/walk_xobject_fontsso fast mode still discovers primary Form XObject CMaps — matching how the page-level font path already worked.Removed
collect_cmaps_from_fonts, a thin wrapper that became dead code oncewalk_xobject_fontscallscollect_cmaps_from_fonts_innerdirectly to pass the propagated flag through.Testing
form_xobject_with_indirect_resources_discovers_tounicode_cmap: builds a minimal in-memory PDF (page → Form XObject → indirect/Resources→ Type0 font with/ToUnicode) and confirmsFontCMaps::from_docdiscovers the CMap. Verified it fails without the fix.form_xobject_indirect_resources_discovered_in_fast_mode_too: same fixture viafrom_doc_pages_fast. Verified it fails without the second part of the fix (thecollect_cmaps_from_xobjectsgating).cargo fmt,cargo test(948 unit tests passing, 2 new),cargo clippy --all-targets -- -D warningsclean.🤖 Generated with Claude Code
Summary by cubic
Fixes garbled/empty text when a Form XObject’s
/Resourcesis an indirect reference by resolving it correctly, and ensures fast mode still discovers primary ToUnicode CMaps on Form fonts. Previously, indirect/Resourcescaused font CMap collection to be skipped; now both inline and indirect dictionaries are handled without changing the fast mode’s skip of expensive TrueType fallback parsing.Review notes
resolve_dict()and uses it for/Resources,/Font, and nested font dict lookups; aligns with existing/XObjectresolution.collect_cmaps_from_xobjectsin both normal and fast paths and threadsskip_truetype_fallbackthroughcollect_cmaps_from_xobjectsandwalk_xobject_fonts.collect_cmaps_from_fontswrapper;collect_cmaps_from_fonts_inneris called directly./Resourcesin normal and fast modes.Written for commit 17e55dc. Summary will update on new commits.