Add periodic cyclic GC to the service loop to stop unbounded memory growth

[SimonHeybrock]

Summary

Mitigation for #1264: detector_data (and, by the same mechanism, every service driving a StreamProcessor) accumulates purely reclaimable cyclic garbage — ~5 self-referential networkx graphs per chunk per job, each transitively holding that chunk's detector arrays — and CPython's count-based generation-2 collector never runs in steady state, so the service grows ~60 MB/cycle until OOM-killed.

This PR makes collection deterministic instead of count-triggered:

• New core/cyclic_gc.py with PeriodicGarbageCollector: a full gc.collect() at a fixed time interval (default 1 s, matching the batch cadence), bounding retained garbage to roughly one cycle's worth.
• gc.freeze() once at loop start, moving process-lifetime service state into the permanent generation so each collection only walks objects allocated since — measured ~0.5 ms per collection instead of ~84 ms (the difference matters against a Timepix3 view update that already costs ~840 ms of the ~1 s budget). The freeze happens before any job starts, so per-job state remains collectable and stopping a job does not leak.
• Collection durations and reclaimed-object counts are logged every 10 minutes (gc_collect_stats), so the predicted upward drift of collection cost as jobs accumulate is observed rather than assumed.
• Enabled by default in Service; collect_garbage=False opts out, and a collector instance can be injected for testing.

This is a symptom fix, as the issue itself insists: it makes the collector run often enough to reclaim garbage that should not be created. It must not close_ #1264 — the root cause (cyclebane's Graph.__setitem__ leaving self-referential intermediates, and ess.reduce's StreamProcessor using the task graph as a per-chunk data container) is tracked there and upstream.

Verification

• Independently re-verified the root cause against the pinned sciline==25.11.1 / cyclebane==24.10.0: one pipeline[key] = value leaves exactly 5 refcount-immortal DiGraphs behind, held only by self-referential cached views (edges, in_edges, out_degree); one gc.collect() frees all of them. The issue's minimal reproduction reproduces: unbounded sawtooth growth without collection, flat RSS (188 MB) with per-cycle collection.
• New unit tests in tests/core/cyclic_gc_test.py cover: cyclic garbage reclaimed by maybe_collect, interval gating (with injected clock), freeze exempting pre-existing objects, and end-to-end through Service._run_loop with the automatic collector disabled — including the control showing the loop leaks cycles without the collector.
• Full fast suite: 4782 passed; the 3 failures (dream/estia workflow roundtrip and reduction tests) are environmental — they need to download calibration files blocked by this sandbox's proxy and fail identically on unmodified main.
• ruff check and ruff format clean.

Refs #1264

🤖 Generated with Claude Code

https://claude.ai/code/session_01EUhFAixW77K2fPL1ySQ5L8

---

Generated by Claude Code

[SimonHeybrock]

Closing without merging.

The two upstream fixes remove the garbage at its source, which makes this mitigation redundant for the leak it was written for: scipp/cyclebane#33 (networkx cached views no longer hold a back-reference, so the discarded graphs are refcount-collectable) and scipp/ess#732 (StreamProcessor feeds values through a slot instead of pipeline[key] = value, so the graphs are not rebuilt per chunk). Either one alone closes #1264; both are close to release.

It is also worth recording that this PR did not fix the production symptom. The staging deploy on 2026-08-31 was still OOM-killed 54 min after the Timepix3 job activated. Its own gc_collect_stats showed reclaimed objects flat while collection duration grew 19 -> 56 ms, i.e. the growth was in live objects, which gc.collect() cannot touch. That turned out to be the unbounded RateAwareMessageBatcher._overflow backlog (#1271) compounded by the O(pixels) per-update floor (#1270).

Beyond redundancy, keeping it has costs:

• It masks the class of bug it defends against. A future cycle capturing a large array becomes slow invisible drift rather than a loud OOM.
• gc.freeze() is process-global state mutated by a library class and never undone. tests/service_test.py starts a real Service, so a unit test permanently freezes everything alive in the pytest worker, and repeated start() calls accumulate.
• The cost argument did not hold in production. The 1 s interval was justified by ~0.5 ms collections after freeze; staging measured 19-56 ms, up to ~5.6% of the cycle budget and rising with uptime.

The one part that earned its place was the diagnostic, not the collection: logging collection duration alongside reclaimed count is what identified the second leak, since rising duration at a flat reclaimed count means the live set is growing. That signal does not need a 1 Hz gc.collect() -- a periodic len(gc.get_objects()) sample in the existing metrics gives the same thing for microseconds. Worth adding separately if we want it.

#1264 stays open for the upstream work.