feat(project-graph): activate semantic extraction on import + LLM-grounded extractor (Phases 1–2, folds #927)#931
Open
seonghobae wants to merge 2 commits into
Open
Conversation
…agged) Wires the existing-but-dead project semantic graph pipeline into the import path: after an imported email commits, its content segments are projected into project_graph_objects via extract_project_semantics + persist_project_graph_ projection. This makes the projects /candidates surface return real data instead of an empty store. Safety: - Gated behind PROJECT_GRAPH_EXTRACTION_ENABLED (default False) -> zero change to existing import behavior until explicitly enabled. - Best-effort: runs AFTER the email is committed, wrapped in try/except with rollback, so a projection failure never fails the email import. - Segments are snapshotted before commit (in-memory, no async lazy-load) to avoid MissingGreenlet; persistence runs after commit when segments have ids. - Workspace scope uses the repository's convention (workspace-<org_id>, or workspace-<user_id> without an org). Tests (tests/test_project_graph_import_wiring.py, 6): real deterministic extractor + mocked persistence — mapping, workspace scoping (org + user fallback), no-op on empty/no-objects, and best-effort rollback on failure. Existing import + project-graph suites unaffected (56 passed). This is Phase 1 of docs/plans/2026-07-06-ai-moat-project-graph-activation.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RkKdtHRLG4wSLh6PVsp8J
…ations Phase 2 of the project-graph activation plan (stacked on the Phase 1 import wiring): upgrades extraction from the deterministic keyword baseline to an LLM-backed extractor that cannot fabricate. - services/project_graph/llm_extractor.py: structured-output extraction over the imported email's content segments. Hard grounding rule: every object must cite content_segment_uids present in the input; objects with unknown, missing, or partial-unknown citations are DROPPED. Unknown object types dropped; confidence clamped to [0,1]; segment text passed as JSON data (prompt-injection hardening, mirrors api/llm.py); base_url goes through the SSRF-validated provider client; segments capped (40 x 2000 chars) to bound tokens. - Selection: PROJECT_GRAPH_EXTRACTOR=keyword|llm (default keyword -> zero behavior change). The llm path reuses the import's OpenAI-compatible provider credentials and falls back to the keyword extractor on any failure, so the projection is never lost to a provider outage. Tests (7): grounded mapping + evidence edges; hallucinated/empty citations dropped; unknown type dropped + confidence clamped; empty segments short- circuit without an LLM call; selection uses llm when configured, falls back to keyword on llm failure, defaults to keyword. Regressions green (18). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017RkKdtHRLG4wSLh6PVsp8J
seonghobae
changed the base branch from
feat/project-graph-import-activation
to
develop
seonghobae
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What (stacked on #927 — Phase 2 of #925)
Upgrades project-graph extraction from the deterministic keyword baseline to an LLM-grounded extractor that cannot fabricate:
content_segment_uids present in the input segments — objects with unknown/missing/partially-unknown citations are dropped. The model cannot introduce uncited domain claims into the graph.object_typedropped;confidenceclamped to [0,1]; titles bounded to the DB column limit.api/llm.py);base_urlgoes through the SSRF-validated provider client; request bounded to 40 segments × 2000 chars.PROJECT_GRAPH_EXTRACTOR=keyword|llm(defaultkeyword→ zero behavior change). Thellmpath reuses the import's OpenAI-compatible provider credentials and falls back to keyword on any failure — the projection is never lost to a provider outage.Verification (local — 7 new tests + 18 regressions green)
grounded mapping + evidence edges · hallucinated/empty citations dropped · unknown type dropped + confidence clamped · empty segments short-circuit (no LLM call) · selection: llm-when-configured / keyword-fallback-on-failure / keyword-default.
Follow-up (Phase 3, per #925)
Corrections → per-tenant eval set + calibrated confidence.
🤖 Generated with Claude Code