[FEATURE] Framework-level logic-change detection for live components

[pyjuan91]

What is the use case?

A durable live source (e.g. OCI Object Storage with a Streaming cursor, or Kafka with committed offsets) can skip its startup full scan (update_all()) on rerun when (a) bootstrap already happened, (b) the stream is durable, and (c) processing logic is unchanged — the durable stream replays the downtime backlog, so a fresh scan is redundant. Condition (c) is the hard part: #2116 ships this for OCI via a user-declared logic_version that the user must bump by hand whenever their transform changes. A forgotten bump silently skips a scan it shouldn't, keeping stale results. That's a deliberate stepping stone, not the safe end state, the framework already fingerprints processing logic for incremental change detection, so it should compute the "did my logic change?" signal itself rather than trust a hand-maintained string.

Describe the solution you'd like

Framework-computed logic-change detection over a live component's subtree, so a durable connector can gate its startup scan without a user-maintained version.

Why this isn't a thin read primitive. Logic-change detection already exists for the synchronous build tree: each #[coco.fn] has a logic fingerprint, per-component deps persist as ComponentMemoizationInfo.logic_deps , reuse is gated by all_contained_with_env(logic_deps), and synchronous child deps roll up to the parent via ComponentRunOutcome.logic_deps. That rollup does not happen across a live subtree: live children build in isolated contexts with parent_context: None, the live op outcome HandleOutcome carries no logic_deps , and a live component resolves its own parent-readiness with an empty ComponentRunOutcome. So a live component's persisted deps cover only its own body (the static mount glue), the user's real per-item transform lives in child components and never enters an aggregate the component can read back on rerun. Detecting "did anything in my live subtree change?" needs the engine to aggregate logic deps across the subtree, which doesn't exist today.

Proposed direction (high level, design-sync-first). At bootstrap, collect the union of logic_deps across the live component's subtree, persist it via the #2078 committed-state primitives (StateKind::Live), on rerun, skip the startup scan iff bootstrap_done ∧ durable ∧ all_contained_with_env(stored_set). This reuses the existing containment check (an edited or deleted function's fingerprint is no longer registered -> not contained -> scan), so a real logic change can't be silently missed and the failure-safe default is "scan." It replaces #2116's declared logic_version with a framework-computed signal.

Two questions I'd like to settle before implementing. I've put my current lean on each, happy to be talked out of either.

1. How to aggregate the subtree's logic deps: a read-side walk of the persisted subtree memo entries after the bootstrap scan, vs. pushing deps up through the live drain path. My lean: the read-side walk. It leaves the live drain/readiness hot path untouched and picks up memoized (cache-hit) children for free, at the cost of coupling to the stable-path prefix layout. The push-up route has to thread a deps channel through the intentionally-decoupled drain path and define a capture window, which is more invasive.

2. Whether the signal must also capture processing-affecting config/args. Today logic_deps covers function code hashes + tracked-context changes, but not a constant arg like chunk_size=512 passed to a child processor, editing it to 1024 moves no registered fingerprint, so a naive aggregate would wrongly skip the rescan. Folding the child's processor_fp in doesn't work either, since for mount_each the per-item value is also a bound arg (value = data the stream replays, constant config = logic). My lean: require processing-affecting config to flow through tracked context, so it's captured for free with no new surface. The concern I'd want your read on: that leaves a residual silent-skip if config is passed as a plain arg, the same failure shape as feat(oci): skip startup full scan on unchanged logic #2116's forgotten version bump, just much smaller. So, is shrinking that hole acceptable for the first cut, or is closing it structurally (separating constant config from per-item value at the API and fingerprinting the config explicitly) the bar?

Additional context

Follow-up to #2017, where @georgeh0 confirmed we'd want framework-level logic-change detection and @badmonster0 greenlit a separate issue to track it. Builds on #2078 (committed-state primitives) and replaces the stepping stone in #2116 (OCI declared-version), its logic_version kwarg becomes redundant and should be removed when this lands.

Scope / non-goals: durability detection stays connector-owned (Kafka offsets / OCI cursor), this is not generic memoization of live components or per-item skip. The framework provides the logic-change signal, the connector keeps owning the durable-stream guarantee and the skip wiring.

I'd like to sync on the two questions above before starting, happy to take the implementation.

---

❤️ Contributors, please refer to 📙Contributing Guide.
Unless the PR can be sent immediately (e.g. just a few lines of code), we recommend you to leave a comment on the issue like I'm working on it or Can I work on this issue? to avoid duplicating work. Our Discord server is always open and friendly.

[georgeh0]

Thanks for the detailed writeup — the framework-computed direction is exactly right. On the two questions:

Q1 (aggregation) — let's go with the write-path direction, not the read-side walk.

The deciding factor is scale. For a durable source over millions of objects, the read-side walk is a second O(fanout) pass over persisted memo entries on every (re)scan — a separate multi-million-row read on top of the scan itself. That won't hold up at the sizes this feature is meant for.

Write-path aggregation avoids that: accumulate the subtree's logic deps as the component processes, rather than re-deriving them afterward. Most of the cost can fold into work the scan already does, and the aggregate is small (bounded by the number of distinct functions, not items), so it stays cheap even at high fanout. Persist it via the #2078 committed-state primitives and gate the skip on all_contained_with_env(...) as you proposed.

I'll leave the detailed design to you — there are a few things worth thinking through (how the aggregate flows up without entangling the live readiness path, how update_full vs. incremental updates each affect it, and how it behaves under cache hits). Happy to sync on the approach once you've sketched it.

Q2 (config/args) — split out to #2126. logic_deps tracks code, not argument values, so a processing-affecting constant passed as an arg (e.g. chunk_size=512) isn't covered by the subtree aggregate alone. That's a separate signal that builds on this one but doesn't block it — details in #2126. This issue can ship with that as a documented known gap.

[pyjuan91]

Hi @georgeh0 , here's the write-path sketch. Working through the cache-hit case you flagged surfaced something that changes the approach, so I'm raising it before building.

Finding: a memoized parent's persisted logic_deps don't include the functions it mounts. (Verified with a small repro.)

Edit only a mounted child's body and rerun → the parent stays a memo hit, the child never re-runs, and its output goes stale. Editing the parent itself does invalidate, so this is specific to the mount boundary.

This also corrects something in my description above: I said sync child deps roll up to the parent via ComponentRunOutcome.logic_deps. They don't; execute_once drains that field before returning, so it's always empty in practice. Only fn calls within a component propagate (propagate_children_fn_logic); mounted child components don't.

Why it matters here: on a rerun scan, an unchanged process_file is a memo hit whose stored
deps are just its own fingerprint, process_chunk's isn't in there. A write-path aggregator
that only unions each hit's stored deps would miss it, and editing process_chunk would
wrongly pass the skip gate. The aggregate is only safe if each stored memo already covers its
whole subtree.

Sketch: the three things you flagged:

• Flowing up without touching live readiness. The aggregation rides the sync build-tree rollup (ComponentRunOutcome / merge_logic_deps) inside update_full's subtree, a separate path from the live HandleOutcome / mark_ready machinery, which stays as-is. Concretely, this means activating the (currently dormant) rollup so a parent's persisted memo carries its mounted children's deps.
• update_full vs. incremental. update_full recomputes the set and replaces it, so fingerprints of removed or edited code drop out. An incremental operator.update extends it with the new item's subtree deps, a streamed item is an existing item on the next restart, so its code has to be in the set. Delete changes nothing. Both persist to the feat(core): add read_committed_state — process_live-callable reader for use_state #2078 Live keyspace.
• Cache hits. Once stored memos cover their subtree, a hit contributes its full set of deps, both to this aggregate and to normal incremental change detection.

One call I'd like from you. Making stored memos subtree-complete is the same change that makes a memoized parent re-run when a mounted child changes, so it closes the staleness above for every memo+mount pipeline, not just live ones (the OCI example included). Two ways to scope it:

1. Fix it in core memoization. Closes the gap everywhere. Cost: editing a deep function now invalidates its memoized ancestors (more correct, but more reprocessing).
2. Limit it to live-aggregating subtrees. Leaves non-live memo behavior unchanged, but the gap stays for non-live memo+mount.

I lean toward (1). It's the more correct behavior, and this feature comes for free with it, but it changes core semantics, so I'd rather you make the call.

[georgeh0]

Hi @pyjuan91 ,

Thanks for flagging the bug in existing regular component logic change detection. Since this is an existing bug, I fixed this directly and just merged #2142

Please go ahead to work on the changes on live components. Thanks!