PDF parsing: pdf-extract has poor table & paper layout recognition, heavy information loss

[plpycoin]

The current PDF parsing relies on the pdf-extract library. It has the advantages of fast speed and concise implementation, but also obvious drawbacks: poor support for table structures and low recognition capability for journal and paper layouts. PDF parsing is the stage where the most information is lost. Are there any improvement plans for this issue going forward?

[lestbet]

I support that idea / request. It would be great to support external Content extraction engines such as docling (https://github.com/docling-project/docling).

[WaylandYang]

Thanks for raising this, and @lestbet for weighing in. You're right that PDF parsing is where we lose the most today: pdf-extract reads the text layer quickly, but tables, multi-column paper layouts and scanned pages come through badly.

We're evaluating several approaches, including Docling, MinerU 2.5 Pro (VLM) and PP-OCRv6, on table structure, paper and journal layouts, and scanned documents. The parser plugs in behind the block model from 0039, so chunking and extraction stay the same whichever one wins.

Better PDF parsing will ship in the official 0.1 release. Thanks, everyone!

[gitvivek14]

thanks ! I'd also recommend looking into OvisOCR2. It has a smaller footprint than MinerU 2.5 Pro and ranks higher on the OmniDocBench (v1.6_full) Leaderboard: OvisOCR2