Skip to content

fix: Form XObject text garbled when /Resources is an indirect reference - #407

Open
MADENIYOU wants to merge 1 commit into
firecrawl:mainfrom
MADENIYOU:fix/form-xobject-indirect-resources
Open

fix: Form XObject text garbled when /Resources is an indirect reference#407
MADENIYOU wants to merge 1 commit into
firecrawl:mainfrom
MADENIYOU:fix/form-xobject-indirect-resources

Conversation

@MADENIYOU

@MADENIYOU MADENIYOU commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Fixes #398.

Root cause

FontCMaps::walk_xobject_fonts (src/tounicode.rs) resolved a Form XObject's /Resources entry with:

if let Ok(form_resources) = stream.dict.get(b"Resources").and_then(Object::as_dict) {

This only handles /Resources as an inline dictionary. Per spec, it may equally be an indirect reference to a dictionary — and when it is, .as_dict() on the Object::Reference fails, so the if let silently doesn't run and the Form's fonts (including any /ToUnicode CMap) are never collected. That can produce garbled/empty Markdown and incorrectly route an otherwise-text-based page to OCR.

Interestingly, /XObject resolution a few lines above this already handled both shapes correctly (Ok(Object::Reference(id)) => ..., Ok(Object::Dictionary(dict)) => ...) — /Resources and /Font just hadn't been brought in line with that.

Fix

Added a shared resolve_dict() helper and used it for every dictionary-or-reference lookup in this function (/Resources, /Font, and the font dict's own entries), matching the pattern already used for /XObject.

Also fixed a related gap: the fast/page-selected CMap path (from_doc_pages_fast) skipped Form XObject font collection entirely (collect_cmaps_from_xobjects was gated behind !skip_truetype_fallback), rather than still collecting cheap primary ToUnicode CMaps while only skipping the expensive embedded-font fallback parsing. Threaded skip_truetype_fallback through collect_cmaps_from_xobjects/walk_xobject_fonts so fast mode still discovers primary Form XObject CMaps — matching how the page-level font path already worked.

Removed collect_cmaps_from_fonts, a thin wrapper that became dead code once walk_xobject_fonts calls collect_cmaps_from_fonts_inner directly to pass the propagated flag through.

Testing

  • New unit test form_xobject_with_indirect_resources_discovers_tounicode_cmap: builds a minimal in-memory PDF (page → Form XObject → indirect /Resources → Type0 font with /ToUnicode) and confirms FontCMaps::from_doc discovers the CMap. Verified it fails without the fix.
  • New unit test form_xobject_indirect_resources_discovered_in_fast_mode_too: same fixture via from_doc_pages_fast. Verified it fails without the second part of the fix (the collect_cmaps_from_xobjects gating).
  • cargo fmt, cargo test (948 unit tests passing, 2 new), cargo clippy --all-targets -- -D warnings clean.

🤖 Generated with Claude Code


Summary by cubic

Fixes garbled/empty text when a Form XObject’s /Resources is an indirect reference by resolving it correctly, and ensures fast mode still discovers primary ToUnicode CMaps on Form fonts. Previously, indirect /Resources caused font CMap collection to be skipped; now both inline and indirect dictionaries are handled without changing the fast mode’s skip of expensive TrueType fallback parsing.

Review notes

  • Adds resolve_dict() and uses it for /Resources, /Font, and nested font dict lookups; aligns with existing /XObject resolution.
  • Calls collect_cmaps_from_xobjects in both normal and fast paths and threads skip_truetype_fallback through collect_cmaps_from_xobjects and walk_xobject_fonts.
  • Removes the thin collect_cmaps_from_fonts wrapper; collect_cmaps_from_fonts_inner is called directly.
  • Adds two unit tests covering indirect /Resources in normal and fast modes.

Written for commit 17e55dc. Summary will update on new commits.

Review in cubic

walk_xobject_fonts resolved a Form XObject's /Resources entry with
`stream.dict.get(b"Resources").and_then(Object::as_dict)` — handling
only the inline-dictionary case. Per spec, /Resources (like most PDF
dictionary-valued entries) may equally be an indirect reference to a
dictionary. When it was, `.as_dict()` on the Reference object failed
silently, and the Form's fonts (including any /ToUnicode CMap) were
never collected — producing garbled or empty text and incorrectly
routing an otherwise-text-based page to OCR.

Added a shared resolve_dict() helper and used it everywhere a
dictionary-or-reference value needs resolving in this function
(/Resources, /Font, and the Font dict's own entries), consistent with
how /XObject was already resolved a few lines above.

Also fixed a related gap the issue flagged: the fast/page-selected
CMap path (from_doc_pages_fast) skipped Form XObject font collection
entirely (collect_cmaps_from_xobjects was only called when
!skip_truetype_fallback), rather than still collecting cheap primary
ToUnicode CMaps while only skipping the expensive embedded-font
fallback parsing. Threaded skip_truetype_fallback through
collect_cmaps_from_xobjects/walk_xobject_fonts so fast mode still
discovers primary Form XObject CMaps.

Removed collect_cmaps_from_fonts, a thin wrapper that became dead code
once walk_xobject_fonts called collect_cmaps_from_fonts_inner directly
to pass the propagated flag.

Fixes firecrawl#398.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 1 file

Shadow auto-approve: would auto-approve. Focused bug fix: resolves indirect /Resources dictionaries so Form fonts' ToUnicode CMaps are collected, and fast mode now still discovers primary CMaps while skipping only expensive fallback. Tests pin both paths; no rollout, contract, or ops change.

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Form XObject text is garbled when /Resources is an indirect reference

1 participant