Add API to supply pre-computed figures/tables and areas to ignore - #1375
Draft
lfoppiano wants to merge 13 commits into
Draft
Add API to supply pre-computed figures/tables and areas to ignore #1375lfoppiano wants to merge 13 commits into
lfoppiano wants to merge 13 commits into
Conversation
lfoppiano
marked this pull request as draft
March 3, 2026 08:04
lfoppiano
force-pushed
the
feature/add-ignore-areas-api
branch
2 times, most recently
from
March 3, 2026 08:05
275a28e to
63f0189
Compare
lfoppiano
force-pushed
the
feature/add-ignore-areas-api
branch
from
June 14, 2026 05:41
20d703a to
647249a
Compare
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| } | ||
| } | ||
| // Check if we're switching blocks | ||
| if (tok.getBlockPtr() != token.getBlockPtr()) { |
| graphicVector = false; | ||
| graphicBitmap = false; | ||
|
|
||
| double pageHeight = block.getPage().getHeight(); |
Introduce a Typed Areas API allowing clients to declare rectangular PDF regions with a type (figure, table, ignore, paratext) for specialized processing. Layout tokens are filtered by these areas: figure/table regions are routed to the dedicated parsers (tables grouped per area), while ignore/paratext regions are excluded from body processing. - New layout types TypedArea and AreaType - Document.filterLayoutTokensByTypedAreas categorises tokens and recomputes block pointers; excluded tokens are dropped from body pieces - typedAreas threaded through GrobidAnalysisConfig and the REST endpoints (header, header-funding, fulltext, fulltext-asset, references) via a parseTypedAreas JSON helper - Documentation (Typed-Areas-API.md, Grobid-service.md, index.md) Squashed from feature/add-ignore-areas-api and rebased onto master, integrating with the concurrent debug-labelling feature.
Signed-off-by: Luca Foppiano <luca@foppiano.org>
lfoppiano
force-pushed
the
feature/add-ignore-areas-api
branch
from
July 21, 2026 15:40
5607763 to
f6f974a
Compare
Signed-off-by: Luca Foppiano <luca@foppiano.org>
Signed-off-by: Luca Foppiano <luca@foppiano.org>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a new typedAreas API parameter to allow callers to supply pre-identified page regions (figures/tables/ignored/paratext) and have GROBID filter tokens accordingly and run specialized figure/table extraction on those regions, with corresponding documentation and unit tests.
Changes:
- Add
typedAreassupport to REST endpoints and propagate it intoGrobidAnalysisConfig. - Implement token filtering + typed-area figure/table processing in the parsing pipeline and ensure TEI serialization doesn’t drop typed-area assets.
- Add new
AreaType/TypedAreacore types, update table completeness behavior, and add unit tests + docs.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| grobid-service/src/main/java/org/grobid/service/process/GrobidRestProcessFiles.java | Thread typedAreas into service processing config for header/fulltext flows. |
| grobid-service/src/main/java/org/grobid/service/GrobidRestService.java | Accept and parse typedAreas multipart field and pass parsed areas to processing. |
| grobid-core/src/main/java/org/grobid/core/layout/TypedArea.java | New value object for page regions with geometry + type and intersection helpers. |
| grobid-core/src/main/java/org/grobid/core/layout/AreaType.java | New enum defining supported region types. |
| grobid-core/src/main/java/org/grobid/core/engines/config/GrobidAnalysisConfig.java | Add typed-areas field + builder hook to carry areas through the pipeline. |
| grobid-core/src/main/java/org/grobid/core/engines/HeaderParser.java | Apply typed-area token filtering prior to header processing. |
| grobid-core/src/main/java/org/grobid/core/engines/FullTextParser.java | Apply typed-area filtering, run typed-area figure/table extraction, and preserve/merge results through annex processing. |
| grobid-core/src/main/java/org/grobid/core/document/Document.java | Add storage for typed areas, excluded tokens, and filtering/splitting utilities. |
| grobid-core/src/main/java/org/grobid/core/document/TEIFormatter.java | Ensure typed-area figures/tables still serialize even when annex text is absent. |
| grobid-core/src/main/java/org/grobid/core/data/Table.java | Mark tables coming from typed areas as TEI-complete even without head/caption. |
| grobid-core/src/test/java/org/grobid/core/layout/TypedAreaTest.java | Unit tests for TypedArea construction, parsing, containment, equals/hash. |
| grobid-core/src/test/java/org/grobid/core/layout/AreaTypeTest.java | Unit tests for AreaType parsing and values. |
| grobid-core/src/test/java/org/grobid/core/data/TableTest.java | Unit tests for TEI completeness behavior when sourced from typed areas. |
| doc/Typed-Areas-API.md | New user-facing documentation for the typed areas request format and behavior. |
| doc/Grobid-service.md | Document typedAreas in service endpoint parameter tables + add a “Typed Areas” section. |
| doc/index.md | Link to the new Typed Areas API documentation page. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+4032
to
+4044
| features.relativeDocumentPosition = featureFactory | ||
| .linearScaling(nn, fulltextLength, NBBINS_POSITION); | ||
|
|
||
| features.relativePagePositionChar = featureFactory | ||
| .linearScaling(mm, 0, NBBINS_POSITION); | ||
|
|
||
| double pageHeight = 1.0; | ||
| if (token.getPage() >= 0 && doc.getPages() != null && token.getPage() < doc.getPages().size()) { | ||
| Page page = doc.getPages().get(token.getPage()); | ||
| if (page != null) { | ||
| pageHeight = page.getHeight(); | ||
| } | ||
| } |
Comment on lines
+517
to
+522
| } else { | ||
| LOGGER.warn("typedAreas should be a JSON array, but received: " + typedAreasJson); | ||
| } | ||
| } catch (Exception e) { | ||
| LOGGER.error("Failed to parse typed areas JSON: " + typedAreasJson, e); | ||
| } |
| | | | | `includeRawCitations` | optional | `includeRawCitations` is a boolean value, `0` (default. do not include raw reference string in the result) or `1` (include raw reference string in the result). | | ||
| | | | | `debugMode` | optional | If `1` or `true`, replaces the response with a `text/plain` dump of the raw CRF/sequence-labelling output of each model invoked during processing. See [Debug raw labelling output](#debug-raw-labelling-output). | | ||
| | | | | `models` | optional | Comma-separated list of model names to include in the debug response (e.g. `segmentation,reference-segmenter,citation`). Only meaningful when `debugMode` is enabled. The pipeline still runs in full; this only filters the response. Unknown model names yield `400`. | | ||
| | | | | `typedAreas` | optional | JSON array specifying areas with coordinates and types for specialized processing (see [Typed Areas](#typed-areas) below) | |
| | `y` | number | Yes | Y-coordinate of the upper-left corner of the area | | ||
| | `width` | number | Yes | Width of the area | | ||
| | `height` | number | Yes | Height of the area | | ||
| | `type` | string | Yes | Area type: `"figure"`, `"table"`, or `"ignore"` | |
Comment on lines
+17
to
+20
| - `POST /api/processHeaderDocument` - Header extraction with typed areas | ||
| - `POST /api/processFulltextDocument` - Full document processing with typed areas | ||
| - `POST /api/processReferences` - Reference extraction with typed areas | ||
|
|
Figures embedded as PDF Form XObjects routinely carry the text of the page they were exported from, clipped away and invisible on screen but fully present in the font stream. Every font-stream extractor picks it up: whole paragraphs and section headings are emitted two or three times, and no downstream model can tell the duplicates from the real body. It affects 60% of a 2,595-document materials-science corpus, so it is not an edge case. pdfalto gains -discardClippedText (dropping glyphs whose glyph box lies entirely outside the current clip path); GROBID passes it whenever pdf.pdfalto.discardClippedText is set, default true. The lin-64 binary is rebuilt from pdfalto fix/discard-clipped-text (691527a). Measured effect on plain GROBID, normalised similarity against JATS body text: Materials Science (2,595 docs) 0.7593 -> 0.9017 Bioinformatics (1,943 docs) 0.8926 -> 0.8963 The materials-science corpus is where figures are most often embedded as PDF, and there the correction is worth +0.14 similarity on its own -- an order of magnitude more than any modelling change we have measured on that corpus. Only the lin-64 binary is rebuilt here; mac-64, mac_arm-64 and lin_arm-64 still carry the unpatched 0.6.2 build and must be rebuilt on those platforms before this can go upstream.
parseTypedAreas logged and carried on. A payload that failed to parse, or that was not a JSON array, produced an empty area list; an entry with an unknown type or a missing field was skipped. Either way the request returned a perfectly ordinary 200 whose content had silently fallen back towards plain GROBID -- partly, or entirely -- with nothing in the response to say so. For batch experiments that is the worst possible failure mode: it is invisible, and it quietly contaminates a whole run. Now strict. An unparseable payload, a non-array payload, or any single unusable entry aborts with HTTP 400 and a message naming the offending indices and reasons (first five, then a count). An absent or empty parameter is still not an error -- it just means no masking. The success path logs accepted counts by type as before. Verified against a running service: not JSON -> 400 typedAreas is not valid JSON: ... JSON object, not array -> 400 typedAreas must be a JSON array, received: ... type: "banana" -> 400 rejected 1 of 1 entries: [0] Unknown area type: banana missing "type" -> 400 rejected 1 of 1 entries: [0] missing required 'type' field missing "y" -> 400 rejected 1 of 1 entries: [0] missing required 'y' field valid area / no parameter -> 200
…nnex
processTypedAreas parked every figure and table built from a detector region in
doc.annexFigures/annexTables, and the merge after annex processing appended them
all back into the annex lists. The annex was only ever a holding pen, but the
TEI writer takes it literally: on the materials-science corpus 94% of a masked
run's figures were serialised inside <back>, against 0.7% for plain GROBID on
the same PDFs. Wrong for every downstream consumer of the TEI, and it made the
"the serialisation is unchanged" claim untrue.
Each typed-area figure and table is now routed by where it actually sits.
A detector region carries page coordinates but no position in the token stream,
and a region that is pure artwork captures no tokens at all, so the decision is
geometric: annexStartPosition() takes the page and y of the first token of the
ANNEX segment, and anything at or beyond that point goes to the annex, anything
before it to the body. No annex, or no usable bounding box, means body -- the
body is where a figure belongs unless we can show otherwise.
One subtlety worth recording: toTEI() is handed the *local* bodyFigures /
bodyTables lists, not doc.getFigures() / doc.getTables(). Updating only the
Document dropped 151 of 174 figures from the output on a 20-document pilot while
appearing to fix the placement. The locals have to be updated too.
Pilot, 20 materials-science PDFs, grouped masks:
figures captioned in <back>
before 174 156 152 (87.4%)
after 174 156 1 ( 0.6%)
plain GROBID (same PDFs) 169 167 1 ( 0.6%)
The figure and caption sets are identical before and after in all 20 documents,
so this relocates figures and loses none. The single remaining annex figure is
in a document with genuine back matter.
Does not deduplicate typed against native figures -- still a separate problem.
spotlessApply on the typed-area validation added in a354506 -- line wrapping only, no semantic change and no import changes. `./gradlew build -x test`, the command CI runs, now passes.
Splits the two failures apart, which a354506 had lumped together. A payload that cannot be read as a list of areas at all -- not JSON, or JSON that is not an array -- now behaves exactly as if the parameter had been omitted: the request proceeds unmasked and returns 200, with the reason logged at ERROR. The caller plainly did not supply a mask list, and failing the whole request buys nothing. A payload that *is* a list but carries an unusable entry still aborts with HTTP 400. That is the case worth failing on: the caller did supply masks, and applying a subset of them is never what they wanted -- it used to return an ordinary 200 having quietly dropped boxes, which is invisible and contaminates a batch run. Verified against a running service, same PDF throughout: not JSON -> 200, 14 figures (= no-parameter case) JSON object, not an array -> 200, 14 figures (= no-parameter case) empty / omitted -> 200, 14 figures array, unknown type -> 400 rejected 1 of 1 entries: [0] Unknown area type: banana array, missing 'type' -> 400 rejected 1 of 1 entries: [0] missing required 'type' field valid single area -> 200, 15 figures
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TBC