Skip to content

feat(parser): add Omnigent support#1167

Open
wesm wants to merge 12 commits into
mainfrom
riordan/feat/omnigent-support
Open

feat(parser): add Omnigent support#1167
wesm wants to merge 12 commits into
mainfrom
riordan/feat/omnigent-support

Conversation

@wesm

@wesm wesm commented Jul 17, 2026

Copy link
Copy Markdown
Member

Adds local Omnigent support. AgentsView reads conversations from Omnigent’s shared chat.db and archives each one as an independent session, including messages, tool activity, reasoning, parent/child relationships, usage, and recorded cost.

Synchronization

Routine watcher and scheduled work is bounded to changed or recent members. Root workspace and branch edits refresh existing subagents through the archive’s indexed parent relationships, so correctness does not require scanning unrelated conversations. Full-container scans are reserved for initialization, schema changes, explicit rebuilds, recovery, and authoritative audits.

Missing conversations become reversible source_missing tombstones instead of being deleted. Unsupported schemas remain nonfatal and preserve archived data across restarts. Failed member updates keep a generation-safe retry until an unscoped pass covers every configured Omnigent root.

Compatibility and safety

The parser supports legacy and split-workspace schemas, workspace-qualified identities, and binary UUIDs. Identity migrations retain trash and permanent-exclusion state, full scans honor cancellation, and authoritative aggregate costs are preserved when model-level costs are absent.

HTTP and SSH remote sync intentionally exclude Omnigent because transcript data shares its SQLite database with authentication state. Remote support remains disabled until AgentsView can produce a fresh allowlisted export containing only parser-required data.

Reviewers should start with internal/parser/omnigent_provider.go, internal/parser/omnigent.go, and the Omnigent paths in internal/sync/engine.go.

@roborev-ci

roborev-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown

roborev: Combined Review (7a96c24)

The changes have two medium-severity correctness gaps in retry handling and Omnigent discovery error propagation.

Medium

  • Retry sources may be cleared without a forced repair parseinternal/sync/container_scheduling.go:509, internal/sync/container_scheduling.go:532
    Retry sources already found through normal or reconciliation discovery are omitted from retrySources, so existing sources are not marked ForceParse. They may then be filtered as unchanged while the retry is cleared, leaving failed message or usage writes unrepaired. Short recovery entries can also be removed before the non-forced source completes.
    Fix: Return all resolved retry paths separately from newly appended sources, then force matching existing sources through parsing.

  • Omnigent discovery suppresses operational database errorsinternal/parser/omnigent_provider.go:216
    All changedMembers errors are silently discarded. The sync pass is consequently recorded as successful and advances its watermark, potentially causing later cutoff-based syncs to exclude unprocessed conversations after a prolonged transient database failure.
    Fix: Propagate operational errors through Provider.Discover; treat only the explicit unsupported-schema error as a clean empty result.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 15m1s

@wesm wesm changed the title feat(parser): add Omnigent support with bounded container sync feat(parser): add Omnigent support Jul 17, 2026
@roborev-ci

roborev-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown

roborev: Combined Review (a197f25)

Changes need revision: three medium-severity scalability and synchronization issues remain.

Medium

  • internal/sync/engine.go:766, internal/parser/omnigent_provider.go:319 — Every Omnigent filesystem event loads all stored source paths, while omnigentStoredSourceMatches returns every container member. A single-conversation change therefore fingerprints and parses the entire archive, leaving the paged hint API unused. Feed Omnigent a bounded, cursor-driven hint page and schedule only changed members plus a bounded tombstone sample. Add a regression proving parse/fingerprint work remains constant as unchanged archive cardinality grows.

  • internal/parser/omnigent_provider.go:208, internal/parser/omnigent_provider.go:547 — Periodic sync hashes the complete SQLite database and reparses every conversation whenever any byte changes, making one-conversation updates proportional to total archive size. Use indexed change discovery and bounded reconciliation pages for scheduled syncs; reserve whole-container parsing for forced rebuilds and schema-generation changes.

  • internal/sync/engine.go:6867 — Failed Omnigent writes demote pw.sess.ID without applying e.idPrefix. During remote sync, this can update the wrong data-version row, leaving prefixed session metadata current while messages or usage remain stale and causing subsequent parses to skip the session as unchanged. Apply applyIDPrefixToID(e.idPrefix, pw.sess.ID) before reading or updating the data version, and add a regression test for a prefixed partial-write failure.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 8m19s

@roborev-ci

roborev-ci Bot commented Jul 17, 2026

Copy link
Copy Markdown

roborev: Combined Review (160b55a)

High-severity correctness issue and two medium-severity scalability regressions remain.

High

  • internal/sync/engine.go:4698 — Missing-session cleanup is source-wide, while the cwd allow-list permits exclusions when any result from the source is allowed. Because an Omnigent container can contain mixed workspaces, a schema migration or combined update can delete archived sessions from filtered workspaces while rejecting their replacements.
    • Fix: Apply the cwd filter to each excluded stored session individually, preserving sessions whose stored cwd is outside the allow-list. Add a mixed-cwd Omnigent migration/deletion test.

Medium

  • internal/sync/engine.go:766, internal/parser/omnigent_provider.go:452 — Every filesystem event still materializes all stored source paths and all live Omnigent member keys. The paginated ListStoredSourcePathHintPage method is unused, so watcher work and memory scale with total archive size.

    • Fix: Provide Omnigent with a bounded hint page using a rotating cursor, then check membership only for that page.
  • internal/parser/omnigent_provider.go:208, internal/parser/omnigent_provider.go:587 — Each periodic sync hashes the entire database; any change reparses and materializes every conversation and message before unchanged results are discarded. A single changed member therefore causes work and peak memory proportional to the full archive.

    • Fix: Use bounded member discovery for ordinary syncs, reserve complete enumeration for ForceFullDiscovery, propagate cancellation through paged parsing, and assert parsed-result cardinality in the scaling test.

Reviewers: 2 done | Synthesis: codex, 11s | Total: 13m27s

@roborev-ci

roborev-ci Bot commented Jul 18, 2026

Copy link
Copy Markdown

roborev: Combined Review (c00b05c)

Changes need revision: four medium-severity synchronization and scalability issues remain.

Medium

  • internal/parser/omnigent_provider.go:140 — The chat.db-* match includes chat.db-shm, which a read connection may update. That event can reopen the database and create a self-sustaining watcher loop. Accept only chat.db and chat.db-wal, explicitly rejecting -shm.

  • internal/sync/engine.go:766, internal/parser/omnigent_provider.go:452 — Every Omnigent filesystem event materializes all stored source paths and live conversation IDs, so event-time memory and query work scale with the total archive size. Use bounded, cursor-paged hint batches and preserve cursor state across sweeps.

  • internal/parser/omnigent_provider.go:325 — The tracker advances checkedAt before member parsing and database writes succeed. Cancellation or failure can therefore remove unwritten members from later changed-path windows. Retain emitted members for retry until writes commit, or advance the tracker only after explicit post-write acknowledgement.

  • internal/parser/omnigent_provider.go:208, internal/parser/omnigent_provider.go:587 — Each scheduled sync hashes the entire chat.db; any change reparses every conversation before unchanged results are discarded. Use bounded incremental discovery and reconciliation for scheduled syncs, reserving full-database parsing for explicit rebuilds or schema changes.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 12m45s

@roborev-ci

roborev-ci Bot commented Jul 18, 2026

Copy link
Copy Markdown

roborev: Combined Review (27d0b98)

Medium findings remain: Omnigent synchronization work is unbounded and failed or canceled members can be skipped.

Medium

  • Unbounded filesystem-event processinginternal/sync/engine.go:766, internal/parser/omnigent_provider.go:454
    Every Omnigent filesystem event loads all archived source paths and materializes all live conversation IDs, so watcher work and memory scale with total archive size. The new paginated hint API is unused. Feed Omnigent a rotating bounded hint page, check only those members with indexed batched queries, and add small-vs-large archive assertions for query/work cardinality.

  • Full polling reparses the entire containerinternal/parser/omnigent_provider.go:55, internal/parser/omnigent_provider.go:569
    Normal full-sync discovery emits the whole container, causing every conversation and item to be parsed whenever the database fingerprint changes—even if the watcher already synchronized the only changed member. ForceFullDiscovery is ignored, and the integration test leaves resultCount unused while asserting only write count. Use bounded member discovery for normal polling; reserve whole-container parsing for initial import, forced resync, or schema rebuild; and assert parsed-result work remains constant across small and large archives.

  • Tracker advances before successful processinginternal/parser/omnigent_provider.go:326, internal/sync/engine.go:4148
    The change-tracker floor advances before member parsing and persistence succeed, while cancellation draining discards pending jobs. Failed or canceled members can fall behind the next timestamp window and remain stale until a later full sync. Retain these members in explicit retry state or advance the tracker only after successful processing, with cancellation and transient-failure regression tests.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 10m8s

@roborev-ci

roborev-ci Bot commented Jul 18, 2026

Copy link
Copy Markdown

roborev: Combined Review (9a8d764)

Changes need revision: three medium-severity scalability and usage-accounting issues remain.

Medium

  • internal/parser/omnigent_provider.go:569 — Any container fingerprint change parses and materializes every conversation and message before discarding unchanged results. A one-session change therefore makes periodic sync scale with the total archive and may cause substantial daemon memory growth. Discover and parse only changed members during normal sync; reserve whole-container parsing for full discovery, migrations, or explicit rebuilds, and reconcile membership in bounded pages.

  • internal/parser/omnigent_provider.go:304 and internal/parser/omnigent_provider.go:436 — Filesystem-event processing is archive-sized. A cold tracker returns the entire container, while warm events scan all stored source hints and live conversations to detect tombstones. Persist or safely seed tracker state, use bounded cursor-based membership reconciliation, and add cardinality tests that measure work performed or rows scanned.

  • internal/parser/omnigent.go:1013 and internal/parser/omnigent.go:1027 — Missing total_cost_usd fields decode to zero and are stored through a non-nil CostUSD pointer, incorrectly treating unknown cost as authoritative $0 and preventing catalog-based token pricing. Track field presence—such as with *float64—so absent costs remain nil while explicit zero costs are preserved; test aggregate and per-model payloads with missing cost fields.


Reviewers: 2 done | Synthesis: codex, 13s | Total: 10m5s

@roborev-ci

roborev-ci Bot commented Jul 19, 2026

Copy link
Copy Markdown

roborev: Combined Review (98e807c)

Medium-severity issues remain in Omnigent background synchronization, primarily unbounded archive work and cursor durability.

Medium

  • Unbounded warm-event reconciliationinternal/parser/omnigent_provider.go:436
    Every warm filesystem event iterates all stored source paths and scans every live conversation into a map, so work and memory scale with the total archive even when only one conversation changed. The cardinality test checks result fan-out, not rows examined. Reconcile deletions through a bounded, paged cursor or bounded periodic reconciliation, and test actual work across small and large archives.

  • Full-archive periodic sync and materializationinternal/parser/omnigent_provider.go:204, internal/parser/omnigent_provider.go:569
    Each periodic sync hashes the entire chat.db; when a change is detected, it materializes every conversation and message before write deduplication. This makes background I/O and peak memory proportional to archive size. Use keyset-paged reconciliation with bounded batches, reserving full traversal for explicit rebuilds.

  • Cursor advances before durable writesinternal/parser/omnigent_provider.go:492, internal/sync/engine.go:4147
    The change-tracker cursor advances after parsing but before results are durably written. Cancellation or write failure can leave the tracker warm while discarding work, causing older cold-container batches to disappear from changed-path sweeps until a periodic full sync. Advance or acknowledge the cursor only after successful archive writes, or retain and requeue unfinished work.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 14m27s

@wesm

wesm commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

Returning to this

@wesm
wesm force-pushed the riordan/feat/omnigent-support branch from 98e807c to caa2a71 Compare July 23, 2026 03:32
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (caa2a71)

Changes need revision: two medium-severity scalability issues cause routine sync work to grow with the entire conversation archive.

Medium

  • internal/parser/omnigent_provider.go:590 — Normal periodic syncs reparse and materialize every conversation whenever chat.db changes. ForceFullDiscovery is ignored, so work scales with the full archive rather than the changed batch. Use bounded per-member discovery for normal syncs, reserving whole-container parsing for cold initialization, schema changes, and forced rebuilds.

  • internal/parser/omnigent_provider.go:320 — Every warm filesystem event loads all stored virtual paths and scans every live conversation through listOmnigentMemberKeys solely to detect deletions, making watcher CPU and memory O(total conversations). Use bounded deletion tracking or defer deletion reconciliation to an explicit full-reconciliation pass.


Reviewers: 2 done | Synthesis: codex, 13s | Total: 8m18s

@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (bf13284)

The Omnigent parser still has three medium-severity correctness and scalability issues despite no identified security exploit path.

Medium

  • internal/parser/omnigent_provider.go:108, internal/parser/omnigent_provider.go:454 — Forced streaming discovery never seeds the change tracker. After startup reconciliation, the first watcher event still sees warm == false and parses the entire container; short-lived sync workers behave similarly. This makes changed-database work scale with archive size and can exhaust daemon memory. Seed or persist high-water cursors after authoritative discovery, and keep cold-worker discovery bounded to changed member sources.

  • internal/parser/omnigent_provider.go:484 — Split-schema incremental discovery only detects newly allocated conversation and item row IDs. Updates to existing conversations, metadata, or agent configuration retain their row IDs and are missed until restart or authoritative audit. Track indexed updated_at changes—preferably with per-workspace cursors—alongside append-only row-ID cursors.

  • internal/parser/omnigent_provider.go:1006 — Whole-container parsing replaces the caller’s context with context.Background() and uses non-context database queries. Cancellation and daemon shutdown therefore cannot interrupt a large parse. Thread the caller’s context through the parsing callback and database queries, and check for cancellation between conversations.


Reviewers: 2 done | Synthesis: codex, 9s | Total: 8m6s

- test(parser): omnigent cardinality regression; docs + frontend accent
- test(parser): use repo-standard synthetic path in omnigent fixtures
- Merge branch 'main' into feat/omnigent-support
- fix(parser): harden omnigent incremental sync
- fix(parser): reconcile omnigent change edge cases
- fix(parser): bound omnigent workspace sync
- fix(sync): reconcile complete container membership
- fix(parser): detect omnigent in-place changes
- fix(sync): cache omnigent container parses
- fix(sync): reject partially versioned batches
- fix(parser): close omnigent consistency gaps
- fix(parser): encode omnigent tool output as JSON
- fix(sync): bound omnigent scheduled discovery
- fix(sync): preserve omnigent retry work
- fix(sync): complete omnigent retry lifecycle
- fix(parser): close omnigent incremental gaps
- fix(sync): bound omnigent recovery work
- fix(sync): harden omnigent recovery cursors
- fix(sync): recover empty omnigent rebuilds
- fix(parser): disambiguate omnigent hash paths
- fix(sync): retry canceled omnigent work
- fix(parser): bound cold omnigent events
- fix(sync): reconcile cold omnigent archives
- fix(sync): continue omnigent reconciliation
- fix(sync): finish omnigent hint sweeps
- fix(sync): preserve omnigent sweep activation
- fix(sync): bound omnigent recovery state
- fix(sync): page omnigent retry overflow
- fix(sync): recover unstored omnigent retries
- fix(sync): complete omnigent recovery sweeps
- fix(sync): bound omnigent tracker state
- fix(parser): page omnigent fast-change window with keyset cursor
- refactor(sync): extract container scheduling behind a provider capability
- refactor(sync): rewrite omnigent sync on standard container model
- fix(sync): bound omnigent event fan-out and prefix failed-write demotion
- test(ssh): stream resolver script on Windows
- fix(parser): support omnigent binary-uuid ids and ignore -shm events
- test(sync): assert omnigent result cardinality, drop dead hint paging
- fix(parser): keep absent omnigent cost NULL for catalog pricing
- fix(sync): bound omnigent incremental discovery
@wesm
wesm force-pushed the riordan/feat/omnigent-support branch from bf13284 to 001e182 Compare July 23, 2026 12:42
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (001e182)

Code is not ready to merge: one medium-severity retry-handling issue could permanently skip failed Omnigent sessions.

Medium

  • internal/sync/engine.go:4609 — The global Omnigent retry is cleared after any non-aborted sync, even when cutoff filtering excluded the failed source or a root-scoped sync covered different Omnigent roots. Because change-tracker cursors advance before parsing, the failed member may never be rediscovered. Track retries per container/root, clear each retry only after that source is successfully processed, and ensure forced retries bypass mtime cutoff filtering.

Reviewers: 2 done | Synthesis: codex, 7s | Total: 10m7s

NilAway cannot prove that the guarded prefix slice is safe when the local slice starts nil. Initialize it explicitly so the analyzer sees the non-nil invariant while preserving index discovery behavior.
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (370e0ba)

High-severity archive data loss and medium-severity incremental synchronization gaps must be fixed before merge.

High

  • internal/sync/engine.go:7235 — A complete Omnigent container parse classifies previously stored but now-absent conversations as parser-excluded. DeleteParserExcludedSessions then permanently deletes their sessions and messages, violating persistent-archive behavior after a cold start, resync, or schema-triggered container parse.
    • Fix: Route inferred missing conversation ownership through the existing source-missing tombstone path. Reserve hard exclusions for explicit identity migrations, such as legacy unqualified IDs.

Medium

  • internal/sync/engine.go:835 — Failed changed-path processing sets omnigentRetryPending, but only full discovery consumes that flag. Because the provider advances its change cursor before parsing and writing, retrying the same watcher path may classify zero sources and falsely succeed, leaving the conversation stale until a later full reconciliation.

    • Fix: Make changed-path retries honor the pending full-retry state by emitting the container/member again, or defer/restore tracker advancement until parsing and writing succeed. Add a watcher-path failure-and-retry test.
  • internal/parser/omnigent_provider.go:504 — Split-schema incremental discovery scans only newly inserted conversation and item row IDs. Updates to existing rows—such as title, updated_at, workspace, branch, model configuration, or session usage—do not advance either high-water mark, so filesystem events emit no member and the archived session remains stale.

    • Fix: Add a bounded cursor/query for updated conversation and metadata rows, or otherwise ensure metadata-only events identify the affected conversation.

Reviewers: 2 done | Synthesis: codex, 13s | Total: 7m43s

Complete container parses infer missing members from stored ownership; those absences must remain revivable archive tombstones rather than parser exclusions. Watcher retries also need an authoritative replay after a failed write advances the provider cursor.

Split schemas now combine indexed single-workspace timestamps with a fixed recent-member window so metadata and usage commits converge without making routine event work scale with the conversation archive.

VALID (fixed): #1, #2, #3
INVALID (dismissed): none
PEDANTIC (skipped): none
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (71c1ad8)

High-severity correctness issue found in Omnigent synchronization; cancellation handling also needs improvement.

High

  • internal/sync/engine.go:7208 — When the final Omnigent conversation is deleted, a full parse returns SkipNoSession with ForceReplace. Because the physical chat.db path is not virtual, all owned sessions become parser exclusions, permanently deleting archived rows and messages instead of marking them source_missing.
    • Fix: Route empty, complete Omnigent container results through sourceMissingMembers, and add a regression test for deleting the final conversation.

Medium

  • internal/parser/omnigent_provider.go:1171 — Full-container parsing ignores cancellation by using context.Background() and never checking the caller’s context while parsing conversations. Large cold syncs or resyncs cannot stop promptly during shutdown or cancellation.
    • Fix: Thread the parse context through container parsing and database queries, use context-aware queries, and check cancellation between conversations.

Reviewers: 2 done | Synthesis: codex, 9s | Total: 13m16s

An empty authoritative database is valid deletion proof, not a parser exclusion. Retaining its stored ownership as revivable tombstones prevents deletion of the final source conversation from erasing archived sessions and messages.

Container scans can be archive-sized, so caller cancellation must reach the shared-container callback, database reads, and per-conversation loop boundaries.

VALID (fixed): #1, #2
INVALID (dismissed): none
PEDANTIC (skipped): none
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (7a0c9b8)

Omnigent support has three medium-severity correctness issues involving deletion-state migration, periodic reconciliation, and retry handling.

Medium

  • internal/parser/omnigent_provider.go:230 — Schema migration unconditionally marks legacy IDs as parser exclusions. For user-trashed or permanently excluded sessions, the old row and messages are deleted while the new workspace-qualified session is imported visibly, bypassing the user’s deletion state.

    • Fix: Maintain an explicit old-to-new identity mapping and transfer or honor trash/exclusion state before deleting the legacy row. Add migration tests for trashed and excluded sessions.
  • internal/parser/types.go:805 — Omnigent does not enable PeriodicReconcile, despite its bounded tracker missing in-place item edits that do not advance updated_at and metadata-only changes in multi-workspace databases. Because scheduled reconciliation only covers opted-in providers, these changes can remain stale.

    • Fix: Add a bounded scheduled authoritative reconciliation mechanism and test it through scheduledReconcileTargets, or revise the documented behavior if reconciliation is intentionally unsupported.
  • internal/sync/engine.go:4624 — A pending Omnigent full retry is acknowledged even when SyncAllSince filters the forced container source out by modification time. No Omnigent work runs, but the retry generation is cleared, potentially leaving a failed member update unrepaired.

    • Fix: Complete the retry only after an Omnigent source survives filtering and finishes successfully, or exempt forced-retry sources from cutoff filtering. Add a regression test with a cutoff newer than the container modification time.

Reviewers: 2 done | Synthesis: codex, 14s | Total: 18m13s

wesm added 2 commits July 23, 2026 15:48
Workspace-qualified identities must retain user trash and permanent exclusion state instead of resurfacing deleted conversations. Explicit migration relationships let cleanup honor that state without rewriting archived primary keys.

Omnigent watcher discovery is intentionally bounded, so scheduled streamed reconciliation remains the authoritative backstop for edits outside its cursors. Pending correctness retries also bypass incremental cutoffs and are acknowledged only after successful work.

VALID (fixed): #1, #2, #3
INVALID (dismissed): none
PEDANTIC (skipped): none
The review checklist was an implementation aid rather than product documentation and should not ship as part of the public feature branch.
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (0b3d7bf)

The Omnigent integration has three medium-severity correctness and data-integrity issues; no concrete security regression was found.

Medium

  • Global retry state can be cleared by an unrelated scoped syncinternal/sync/engine.go:4624
    A failed sync under root A can be forgotten when a successful changed-path or root-scoped sync processes only root B. Because root A’s change cursor has already advanced, its missed updates may remain unrepaired until periodic reconciliation. Track retry state per root/container, or acknowledge the global retry only after an unscoped pass covering every configured Omnigent root.

  • Generic remote archiving can produce an inconsistent SQLite snapshotinternal/parser/types.go:811
    Marking Omnigent file-backed causes its entire root—including the live WAL-mode chat.db and sidecars—to be copied independently. A checkpoint between copies can yield an inconsistent snapshot or omit committed conversations. Add Omnigent-specific HTTP and SSH archive handling that uses SQLite’s online backup API and exports only the resulting standalone chat.db.

  • Aggregate authoritative cost is discarded when by_model existsinternal/parser/omnigent.go:1152
    Payloads with per-model token breakdowns but only a top-level total_cost_usd are catalog-priced instead of using Omnigent’s recorded cost. Reconcile the aggregate with the model breakdown, allocating the authoritative total across model events when per-model costs are absent.


Reviewers: 2 done | Synthesis: codex, 12s | Total: 13m8s

A failed member parse can advance one root's bounded cursor, so unrelated scoped work must not acknowledge the global replay generation. Only an unscoped pass provides the coverage needed to clear that recovery state.

Remote copies of a live WAL database also need transactional snapshot semantics. Exporting one online-backup database avoids inconsistent main/WAL combinations and keeps unrelated Omnigent state out of HTTP and SSH archives. Recorded aggregate usage cost remains authoritative when Omnigent provides model token breakdowns without model costs.

VALID (fixed): #1, #2, #3
INVALID (dismissed): none
PEDANTIC (skipped): none
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (c636485)

Remote sync has one high-severity credential exposure and two medium-severity correctness/performance issues.

High

  • Omnigent authentication secrets are exported by remote syncinternal/remotesync/archive.go:130

    writeOmnigentChatDBSnapshot streams a page-for-page backup of chat.db, which includes password hashes and redeemable invite or magic-login tokens. Anyone with an AgentsView bearer token could download authentication state for the separate Omnigent service.

    Generate a fresh SQLite export containing only parser-required tables and columns, or exclude Omnigent from remote sync until sanitized exports are supported. Do not sanitize a copied database by dropping tables, since deleted SQLite pages may retain secrets.

Medium

  • Restart cache hits leave the change tracker coldinternal/parser/omnigent_provider.go:527, internal/sync/engine.go:7056

    A persisted whole-container cache hit bypasses parseContainer, so the first subsequent filesystem event parses every conversation and makes watcher work scale with archive size.

    Initialize or restore the tracker while validating cached containers. Add a restart regression showing that one changed conversation results in bounded parsing for both small and large archives.

  • Omnigent snapshot failures incorrectly report successful SSH syncinternal/ssh/transfer.go:171

    If Python/SQLite backup support is unavailable or backup creation fails, Omnigent’s only data source is omitted while the SSH command still succeeds, resulting in a successful sync that imports no Omnigent sessions.

    Make Omnigent snapshot failures fatal while preserving warning-only behavior for optional Hermes snapshots. Cover missing-Python and backup-error cases.


Reviewers: 2 done | Synthesis: codex, 13s | Total: 17m54s

Omnigent stores transcript data beside authentication credentials in the same SQLite database. A page-level backup cannot provide a safe remote artifact, so HTTP and SSH sync must exclude the provider until a fresh allowlisted export schema exists. This also makes snapshot-tool failures irrelevant instead of allowing successful empty imports.

Persisted container cache hits still need to restore the bounded in-memory change cursors after process restart. Rebuilding only cursor tails, workspace bounds, and the fixed recent-member window preserves watcher cardinality without reparsing the archive; a fingerprint recheck closes the commit race during restoration.

VALID (fixed): #1, #2, #3
INVALID (dismissed): none
PEDANTIC (skipped): none
@roborev-ci

roborev-ci Bot commented Jul 23, 2026

Copy link
Copy Markdown

roborev: Combined Review (abbb14d)

Medium-severity issues remain in Omnigent reconciliation despite remote-sync credential handling being clean.

Medium

  • internal/parser/omnigent_provider.go:128 — Every scheduled reconciliation enumerates all conversations via DiscoverEach, then reparses unchanged members because the source fingerprint differs from the stored semantic hash. This makes periodic background work scale with the entire archive, violating the bounded-background-work requirement. Keep scheduled discovery on bounded change cursors and reserve archive-wide semantic reconciliation for explicit resync/manual audits, or persist a comparable freshness value that lets unchanged members skip parsing.

  • internal/parser/omnigent_provider.go:201 — Forced streaming discovery treats ErrOmnigentUnsupportedSchema as a hard failure, while ordinary discovery handles it as a non-destructive unsupported-source outcome. An intentionally unsupported database can therefore make every scheduled or watcher-recovery reconciliation fail indefinitely. Return a non-authoritative, nonfatal result that preserves archived sessions, and add a reconciliation test covering unsupported schemas.


Reviewers: 2 done | Synthesis: codex, 8s | Total: 12m0s

Scheduled reconciliation must stay proportional to the changed member batch instead of reparsing every conversation in the shared database. Full membership scans remain available to archive audits and forced recovery, where their deletion proof is intentional.\n\nUnsupported Omnigent schemas cannot safely prove that archived conversations disappeared. Treating that state as non-authoritative lets healthy reconciliation work finish while preserving the existing archive.
@roborev-ci

roborev-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (3a3e106)

The Omnigent integration has two medium-severity synchronization correctness issues; no high or critical findings were reported.

Medium

  • Unsupported schemas alternate between clean skips and sync failuresinternal/parser/omnigent_provider.go:1146
    Unsupported schemas are initially converted to SkipUnsupportedSource and skip-cached, but a subsequent cache hit restores the same condition as an error. Treat ErrOmnigentUnsupportedSchema as a non-error during cache restoration, or exclude unsupported outcomes from the skip cache. Add a consecutive-sync regression test.

  • Root metadata changes do not update dependent subagent sessions incrementallyinternal/parser/omnigent_provider.go:569
    Incremental discovery emits only conversations whose own rows changed. Children inheriting workspace and branch metadata from a changed root retain stale Cwd, project, and branch values until the daily authoritative audit. Include dependent conversations through an indexed, bounded lookup and test root workspace/branch updates with existing children.


Reviewers: 2 done | Synthesis: codex, 11s | Total: 12m10s

Unsupported parser outcomes may be persisted as clean skip entries, so a cold tracker must preserve that result instead of turning restart validation into a failure.

Root workspace and branch changes also affect archived descendants even when their own source rows are unchanged. Resolving those virtual sources through the archive’s indexed parent relationships keeps incremental correctness proportional to the affected subagent tree.
@roborev-ci

roborev-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (7b130fa)

The Omnigent integration has four medium-severity synchronization and tracker issues that can cause redundant archive-wide parsing, stale inherited metadata, or delayed updates.

Medium

  • internal/sync/engine.go:3128 — General watch-root reconciliation always forces full Omnigent discovery. Startup reconciliation reparses every conversation without seeding the bounded change tracker, so the first watcher event after restart may trigger another full-container parse. Reserve full discovery for archive audits/resyncs; restore ordinary reconciliation from the persisted whole-container cache and use bounded discovery.

  • internal/sync/engine.go:1106 — Descendant expansion for inherited workspace and branch metadata is limited to changed-path classification. SyncAll and scheduled ReconcileProviderRoots discovery only parse the changed root conversation, leaving descendants with stale inherited CWD, project, or branch data after missed events. Apply bounded descendant expansion to regular and streaming Omnigent discovery paths and cover scheduled reconciliation with tests.

  • internal/sync/engine.go:4712 — An Omnigent failure sets a global full-retry flag that only a successful unscoped SyncAll clears. Watcher retries and scheduled provider reconciliation cannot clear it, potentially causing every subsequent watcher event to parse the entire archive until restart or manual sync. Track retry state per container/root and clear it after successful coverage, or make the scheduled pass execute and acknowledge the pending all-root retry.

  • internal/parser/omnigent_provider.go:701 — Incremental updates advance row cursors without updating singleWorkspace or workspaceID. If initialization occurs against an empty split-schema database, the tracker can remain permanently classified as not single-workspace; after the 15-minute recent-member window, metadata-only updated_at changes may go undiscovered until the daily audit. Model empty, single-workspace, and multi-workspace states separately, transitioning an empty tracker when new conversations appear.


Reviewers: 2 done | Synthesis: codex, 15s | Total: 11m43s

wesm added 2 commits July 23, 2026 22:51
Replace the Omnigent-named engine machinery from the reconciliation fix
series with agent-agnostic equivalents:

- Drop the forceFullOmnigent parameter threaded through the three
  reconciliation signatures; whether a pass demands complete coverage is
  derived from the existing scope (unscoped means authoritative).
- Add SourceCapabilities.IncrementalDiscoveryCursor, declared by the
  Omnigent provider, and key the forced-full-discovery retry state per
  agent on it instead of hardcoding parser.AgentOmnigent in the request,
  query, and completion paths.
- The deletion audit constructs providers with ForceFullDiscovery set
  unconditionally: it only runs after an authoritative pass, so any
  cursor-based provider must resolve sources with its cursor bypassed.
Trae and Omnigent were excluded from remote sync artifacts through a
duplicated resolveAgentHasOnDiskSource short-circuit plus eight hardcoded
agent identity checks across the archive, manifest, delta-root, and tar
seams. Declare the exclusion once as AgentDef.RemoteSyncExcluded with the
rationale on each registry entry, and have every seam consult it, keeping
the fail-closed defense-in-depth checks.

Behavior change: the defense-in-depth seams previously named only
Omnigent; they now also reject Trae, which could never legitimately reach
them because target resolution already excludes it.
@roborev-ci

roborev-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (7f5a386)

Changes need fixes: three medium-severity correctness issues were identified.

Medium

  • internal/parser/omnigent_provider.go:1172 — Cached restoration breaks old schemas. Cached-state restoration always queries workspace_id, which is absent from the supported old-generation schema. After restart, restoring a valid skip-cache entry for an old-schema container fails instead of accepting the cached source. Restore row-ID/workspace cursors only for split-metadata schemas; old schemas should restore only the timestamp cursor or use schema-appropriate queries.

  • internal/sync/engine.go:1030, internal/sync/engine.go:3551, internal/sync/engine.go:4712 — Discovery retries can cause repeated archive-wide parsing or remain unhandled. Changed-path processing honors a pending retry but never clears it, so subsequent database events can repeatedly force full parsing. Provider-scoped scheduled reconciliation ignores the retry and may leave the failed source unvisited. Honor retry generations during scheduled reconciliation and clear the matching generation after a successful forced-full pass covering all configured roots.

  • internal/sync/engine.go:12864 — Single-session resync bypasses identity-migration deletion policy. Single-session resync does not call applyProviderIdentityMigrationPolicies. During Omnigent legacy-to-qualified ID migrations, this can delete a trashed legacy row while recreating or retaining its qualified replacement, losing the user’s deletion state. Apply identity-migration policies immediately after processFile, before exclusions or writes, consistent with the batch path.


Reviewers: 2 done | Synthesis: codex, 10s | Total: 13m13s

@roborev-ci

roborev-ci Bot commented Jul 24, 2026

Copy link
Copy Markdown

roborev: Combined Review (7f5a386)

Omnigent integration has three medium-severity synchronization reliability and scalability issues.

Medium

  • internal/sync/engine.go:1106 — Changed-member discovery advances its cursor before descendant expansion. If expandOmnigentInheritedMetadataSources fails, discovered sources are discarded without a full-discovery retry, potentially leaving changes unsynced until an authoritative audit or restart. Call requestFullDiscoveryRetry(AgentOmnigent) on expansion failure, or defer cursor advancement until classification succeeds.

  • internal/sync/engine.go:7427 — Complete container parsing relies on providerSourceSessionOwnershipsForForceReplace, which logs and suppresses archive lookup errors. The parse may then be treated as complete and cached without tombstoning missing conversations. Propagate ownership lookup errors so processing fails, remains uncached, and schedules a forced-full retry.

  • internal/sync/engine.go:1030 — A successful watcher-driven forced retry does not clear discoveryRetries; only an unscoped full sync clears it. After a transient failure, every Omnigent event—including events from unrelated configured roots—can trigger whole-container discovery, causing per-event work to scale with archive size. Track retry generations per container/root and acknowledge each after a successful complete retry for that scope.


Reviewers: 2 done | Synthesis: codex, 8s | Total: 8m46s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants