Repository navigation
feat: proposal-aware amendments — find, confirm, parse, reference - #3
Conversation
Approved brainstorm output for the primary field need: amending existing proposals. Hybrid intent detection with explicit pick-list confirmation, synchronous disambiguation (no pending state), Gemini multimodal PDF parsing with visible provenance and never-invent gaps, Amendments tab columns K–P, Original/This amendment/Grand total presentation, and a kind=new regression guarantee for the phase-1 flow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Companion plan for the approved phase-2 design: extraction intent/hint growth, amps-unique poles inference, Amendments K–P with legacy-row compatibility, the proposals module (Drive search/fetch + Gemini multimodal parse), the three-way intake route, pick-list and provenance UI, and the live Rasmus smoke as the finish line. Full code per task, TDD throughout, single checkpoint push at the CR. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Intent and proposal hint feed the downstream proposal search, and poles become optional so "2 30 amp breakers" doesn't force an invented pole count. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Deterministic inference only when unambiguous, everything else stays UNMATCHED. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Legacy 10-column rows still read correctly: from_row() pads to 16 columns and defaults a blank kind to "new", so pre-existing sheet data keeps working without a migration. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds `app/proposals.py`: searches the shared Skipta/ Drive tree for a customer's proposal, ranking the customer-named folder first and then scoring files within each folder by filename token overlap against a hint string, capped at 5 candidates. `fetch_pdf` returns uniform PDF bytes regardless of source — Google Docs are exported to PDF, native PDFs are downloaded as-is — and rejects anything over the 10 MB fetch cap with `ProposalTooLarge`. `build_drive` is restored to `google_clients.py` to supply the injected Drive v3 service. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Strict schema with nullable total and empty-string defaults, same fallback chain as extraction. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…etadata Task 4 review caught that native Google Docs usually omit Drive size metadata, so the pre-download 10MB cap silently never fired for them. The actual bytes are now the authority: both download paths run through a post-download check, with a regression test using a size-less Doc record whose export exceeds the cap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`POST /api/v1/amendments` now resolves amend intent synchronously with no row written until the ambiguity clears: an amend note with no chosen file returns 200 `choose_proposal` candidates for the operator to pick from, leaving the sheet untouched. `proposal_file_id` distinguishes "not yet answered" (absent/None) from "answered — proceed as new" (empty string) from "answered — here's the proposal" (a Drive file id), so a resolved amend fetches and parses the PDF and writes `kind="amend"` plus its provenance fields, while a declined amend falls through to a plain `kind="new"` row. Fetch/parse failures on the resolved path surface as an honest 502 rather than a silent fallback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
DOM-built candidate buttons replace the intake form's single-shot fetch, provenance names the source file on the signing page and PDF, and the grand-total row appears only when the original proposal's total was actually parsed. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ead errors Task 6 review flagged that the fetch/parse-failure → 502 path was verified only by code reading. A monkeypatched fetch_pdf raising ProposalFetchError now pins both halves of the contract: the status code and the empty Amendments store. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Names the two first-class intake flows with their canonical example sentences and fixes the intro's PDF destination (GCS archive, not Drive — stale since the pivot). Doc links now cover both phases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughSummary by CodeRabbit
WalkthroughAdds amend-vs-new extraction, Drive-backed proposal lookup and parsing, expanded amendment persistence, revised pricing for missing breaker poles, updated amendment routing/UI, and matching documentation and tests. ChangesProposal-Aware Amendments Feature
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Pull request overview
Adds a “proposal-aware amendment” flow alongside the existing “new work order” flow: extraction can detect amend intent + proposal hint, the API can synchronously return a proposal pick-list, and resolved amendments fetch/parse the chosen proposal PDF to populate provenance fields and totals-ladder rendering.
Changes:
- Extend extraction/pricing/data model to support amend intent, optional breaker poles, and new Amendments-sheet columns (kind + proposal provenance fields).
- Add Drive proposal discovery + capped PDF fetch + Gemini multimodal proposal parsing (facts never invented).
- Update intake + signing/PDF templates to support choose-proposal UX and show provenance + Original/This/Grand totals ladder (when original total is present), plus add/extend tests.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
app/extraction.py |
Adds intent, proposal_hint, and optional poles to extraction + schema/prompt updates. |
app/pricing.py |
Supports amps-unique matching when breaker poles are unspecified. |
app/amendments.py |
Expands Amendments row schema to A–P with kind + proposal provenance fields and legacy-row padding. |
app/proposals.py |
New module for Drive proposal search, capped PDF fetch, and multimodal proposal parsing. |
app/google_clients.py |
Restores/introduces build_drive() client creation. |
app/main.py |
Adds 3-way intake behavior (choose_proposal vs new vs resolved amend) and pass-2 proposal parsing integration. |
app/templates/index.html |
Adds choose-proposal button UI and “proceed as new” option in the intake page. |
app/templates/sign.html |
Shows provenance block + totals ladder for amendments. |
app/templates/amendment_pdf.html |
Mirrors signing-page provenance + totals ladder in the rendered PDF. |
app/static/skipta.css |
Styles candidate pick buttons and provenance block. |
tests/conftest.py |
Extends fakes/overrides to include Drive + proposal parsing facts. |
tests/test_extraction.py |
Tests amend intent/proposal hint extraction + optional poles validation. |
tests/test_pricing.py |
Tests amps-unique matching and ambiguity behavior for unspecified poles. |
tests/test_amendments.py |
Tests roundtrip of new proposal fields + legacy 10-column compatibility. |
tests/test_proposals.py |
Tests proposal search ordering, fetch caps, and multimodal parse behavior. |
tests/test_routes_amend.py |
Tests all three intake response shapes and signing-page provenance/grand-total behavior. |
README.md |
Documents the two supported flows and links phase-2 docs. |
docs/plans/2026-07-02-skipta-proposal-amendments-design.md |
Phase-2 design spec added. |
docs/plans/2026-07-02-skipta-proposal-amendments-plan.md |
Phase-2 implementation plan added. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 9
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@app/amendments.py`:
- Around line 23-42: The `AmendmentRecord.to_row` and `AmendmentRecord.from_row`
methods are tightly coupled to the dataclass field order through hardcoded
indices, which makes the row schema fragile. Refactor these methods to use an
explicit field-to-column mapping (or named column constants) instead of
`row[6]`, `row[13]`, `padded[6]`, `padded[10]`, and `padded[13]`, so the
serialization logic in `AmendmentRecord` stays correct even if fields are added
or reordered. Ensure the mapping is defined once and used consistently by both
methods.
In `@app/main.py`:
- Around line 139-153: The amend flow in the main request handler accepts any
client-supplied proposal_file_id and fetches proposal data directly, which can
bypass the candidate lookup. In the branch that handles payload.intent ==
"amend", re-verify body.proposal_file_id against the proposals.search_proposals
candidate set for the current customer_name and proposal_hint before calling
proposals.fetch_pdf or drive.files().get, and reject the request if the ID is
not one of the returned proposal IDs.
- Around line 146-153: The amend-path in main.py has a redundant
drive.files().get metadata lookup after proposals.fetch_pdf, and that extra
Drive call can raise an unwrapped error outside the existing 502 handling.
Either remove the duplicate lookup by having proposals.fetch_pdf return the
proposal name alongside the PDF bytes, or ensure the proposal_name fetch is also
wrapped in the same try/except so all Drive API failures on the amend path are
converted into the existing HTTPException with status 502. Reference the amend
branch in main and the proposals.fetch_pdf flow when updating the exception
coverage.
In `@app/proposals.py`:
- Around line 41-56: The search_proposals flow is making one sequential Drive
files.list call per folder, which scales poorly as customer folders grow. Update
search_proposals to avoid always scanning every folder synchronously: prefer the
matched customer folder first and only expand to other folders if needed, or
fetch children in parallel with bounded concurrency. Keep the Candidate ranking
behavior in place, but make sure the folder loop and child lookup logic in
search_proposals are changed to reduce latency for choose_proposal.
- Line 43: The proposal lookup in proposals.py only reads the first page from
drive.files().list and ignores nextPageToken, so results past the 50-item
pageSize are missed. Update the folder and file listing logic around the
drive.files().list calls in the proposal search flow to loop through all pages
until nextPageToken is exhausted, accumulating results before searching. Keep
the fix localized to the listing code used by the search path so both
customer-folder and folder-file lookups return complete results.
- Around line 60-62: The size check in the proposal fetch flow is doing an
unguarded int() cast on Drive metadata before the error-mapping try/except, so a
non-numeric size can escape as a bare ValueError. Move the size parsing and
comparison into the same guarded block used by the proposal retrieval logic in
app/proposals.py, and make the code around drive.files().get(...) and
ProposalTooLarge/ProposalFetchError/ProposalParseError handle malformed size
values by raising one of the mapped proposal exceptions instead of letting
ValueError escape.
- Around line 50-54: The candidate lookup in proposals search is too broad
because the query in the folder child scan only filters out folders, so
non-proposal files can be scored and later treated as PDFs. Tighten the filter
in the proposal candidate flow around the folder child fetch and Candidate
creation so only proposal-like MIME types are included (for example PDFs and
other approved document types), using the existing proposal search logic in
app/proposals.py. Keep the restriction in the same place as the files().list
query and the downstream scoring that builds Candidate entries so fetch_pdf and
parse_proposal are only reached for valid proposal files.
In `@app/templates/index.html`:
- Around line 23-24: The submitAmendment flow in the index.html template only
handles non-OK HTTP responses and leaves fetch rejections unhandled, so a
network failure can strand the UI on “Working…”. Update the submitAmendment
logic around the fetch("/api/v1/amendments", ...) call to catch rejected fetch
promises, clear or replace the pending state, and surface a user-visible error
message in result.textContent alongside the existing resp.ok handling.
In `@docs/plans/2026-07-02-skipta-proposal-amendments-design.md`:
- Around line 49-52: The markdown table entry for `ProposalFacts` is splitting
because of the unescaped `float|None`, and the `parse_proposal` contract is out
of sync with `app/proposals.py`. Update the `parse_proposal(model_factory,
model_names, pdf_bytes)` spec to match the implementation’s keyword-only
`pdf_bytes` ordering, and escape the pipe in `original_total: float|None` so the
table renders correctly.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 99d57ca6-c995-4c71-a088-c2c34f647d37
📒 Files selected for processing (19)
README.mdapp/amendments.pyapp/extraction.pyapp/google_clients.pyapp/main.pyapp/pricing.pyapp/proposals.pyapp/static/skipta.cssapp/templates/amendment_pdf.htmlapp/templates/index.htmlapp/templates/sign.htmldocs/plans/2026-07-02-skipta-proposal-amendments-design.mddocs/plans/2026-07-02-skipta-proposal-amendments-plan.mdtests/conftest.pytests/test_amendments.pytests/test_extraction.pytests/test_pricing.pytests/test_proposals.pytests/test_routes_amend.py
…, storage-neutral copy Four accepted findings from the PR #3 review round (six deferred as post-MVP hardening, one rejected on threads): - Candidate search now asks Drive for PDFs and Google Docs only — previously any non-folder file (a jobsite photo, a Sheet) could reach the pick-list and be fed to Gemini as bogus PDF bytes. The fake honors the mime clause and a photo-exclusion test pins it. - The post-fetch Drive name lookup sat inside the try but outside the 502 mapping, so a transient HttpError there surfaced as a raw 500; a broad arm now keeps the amend path's honest-502 contract, pinned by a second-call-fails test. - Signing-page copy no longer claims PDFs live in Drive (they are GCS archive links since the pivot). - Design-doc table cell escaped (MD056) and parse_proposal signature aligned with as-built. Also renames the short-row test to say what it actually pins: the Sheets API truncates trailing empty cells on every read, so short rows are normal current-day behavior, not legacy support. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
♻️ Duplicate comments (2)
app/proposals.py (2)
41-59: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winPagination still unhandled on
files.list().Both folder (Line 43) and child-file (Line 54) listings cap at
pageSize=50without readingnextPageToken. Folders or files beyond 50 remain invisible tosearch_proposalswith no error surfaced. This was flagged in a prior review and remains unaddressed.🔧 Proposed fix
- folders = drive.files().list(q=folder_query, fields="files(id, name)", pageSize=50).execute().get("files", []) + folders = _list_all(drive, folder_query, fields="files(id, name)")Add a small pagination helper and reuse it for both listings.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/proposals.py` around lines 41 - 59, `search_proposals` still only reads the first page from both `drive.files().list()` calls, so folders and child files past 50 are skipped. Add a small pagination helper that repeatedly follows `nextPageToken` and returns all files, then use it for both the folder lookup and the per-folder child lookup in `search_proposals` so the ranking logic works across all pages.
63-70: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick winUnguarded metadata access before the error-mapping
tryblock.
int(meta.get("size") or 0)(Line 64) andmeta["mimeType"]/meta["name"](Lines 68-70) all execute before thetryat Line 71. A non-numericsizeor missingmimeType/namekey raises a bareValueError/KeyErroroutsideProposalTooLarge/ProposalFetchError, bypassingmain.py's 502 mapping and surfacing as an unhandled 500. This was flagged previously and remains unaddressed.🔧 Proposed fix
meta = drive.files().get(fileId=file_id, fields="name, mimeType, size").execute() - if int(meta.get("size") or 0) > MAX_PDF_BYTES: - raise ProposalTooLarge(f"{meta['name']} exceeds {MAX_PDF_BYTES} bytes") - request = ( - drive.files().export_media(fileId=file_id, mimeType="application/pdf") - if meta["mimeType"] == DOC_MIME - else drive.files().get_media(fileId=file_id) - ) try: + name = meta.get("name", file_id) + try: + size = int(meta.get("size") or 0) + except (TypeError, ValueError): + size = 0 + if size > MAX_PDF_BYTES: + raise ProposalTooLarge(f"{name} exceeds {MAX_PDF_BYTES} bytes") + request = ( + drive.files().export_media(fileId=file_id, mimeType="application/pdf") + if meta.get("mimeType") == DOC_MIME + else drive.files().get_media(fileId=file_id) + ) result = request.execute()🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@app/proposals.py` around lines 63 - 70, The metadata handling in proposals.py is still happening before the error-mapping try block, so bad or missing Drive fields can raise bare ValueError/KeyError instead of ProposalTooLarge or ProposalFetchError. Move the size/name/mimeType access in the proposal fetch flow into the guarded section around the file download logic, and make sure the code path in the proposal retrieval function uses explicit validation for meta["size"], meta["mimeType"], and meta["name"] before building the request.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@app/proposals.py`:
- Around line 41-59: `search_proposals` still only reads the first page from
both `drive.files().list()` calls, so folders and child files past 50 are
skipped. Add a small pagination helper that repeatedly follows `nextPageToken`
and returns all files, then use it for both the folder lookup and the per-folder
child lookup in `search_proposals` so the ranking logic works across all pages.
- Around line 63-70: The metadata handling in proposals.py is still happening
before the error-mapping try block, so bad or missing Drive fields can raise
bare ValueError/KeyError instead of ProposalTooLarge or ProposalFetchError. Move
the size/name/mimeType access in the proposal fetch flow into the guarded
section around the file download logic, and make sure the code path in the
proposal retrieval function uses explicit validation for meta["size"],
meta["mimeType"], and meta["name"] before building the request.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: 315c383e-0049-4325-89e0-81ddce7c6a10
📒 Files selected for processing (7)
app/main.pyapp/proposals.pyapp/templates/sign.htmldocs/plans/2026-07-02-skipta-proposal-amendments-design.mdtests/test_amendments.pytests/test_proposals.pytests/test_routes_amend.py
Summary
docs/plans/2026-07-02-skipta-proposal-amendments-{design,plan}.md): "Amend Rasmus' SPAN panel proposal…" now finds the proposal in the shared Drive tree, confirms the match with the tech, reads the PDF with Gemini, and produces an amendment that references the original.choose_proposalpick-list (customer-folder-first, hint-token-scored, max 5, plus "None of these → proceed as new work order") — no row is written until the tech resolves.proposal_file_idabsent → pick-list; empty →kind=new; value → fetch + parse +kind=amend.Part.from_data) with a never-invent schema (required: [], nullable total): missing facts render "not found in document" and the Grand total row appears only when the original total was actually parsed. Signing page and PDF gain the provenance block naming the source file and the Original / This amendment / Grand total ladder;kind=newrenders byte-identically to phase 1 (existing suites are the regression net).kind, proposal_file_id, proposal_name, original_total, customer_email, customer_address); legacy 10-column rows read back askind=new.build_drivereturns; consumer-SA writes remain impossible and unneeded — PDFs stay in the GCS archive).Test plan
cigreen: 53-test suite (52 + WeasyPrint render active on ubuntu), ruff clean.kind=amendrow. Negative checks: nonexistent doc → empty pick-list + proceed-as-new; new-intent sentence → no pick-list.Related