Skip to content

fix(mir-fork): per-child private-memory budget and fork-time admission - #10711

Merged
ormastes merged 1 commit into
release/1.0.0from
work/mir-fork-child-budget
Nov 21, 2026
Merged

ormastes merged 1 commit into
release/1.0.0from
work/mir-fork-child-budget

Conversation

@ormastes

@ormastes ormastes commented Nov 21, 2026 •

Copy link
Copy Markdown
Owner

Problem

PR #10663's batching did not make fork mode fit the memory cap. In run p4-cli-f24 (2026-11-22, cgroup cap 100G, FORK=5, batch=200) the processes measured coordinator 18G, root 25G, and children 26-29G RSS. About 7 minutes after the round-0 fork, one child reached 52.5G anon-rss and the memcg OOM killer ended the build. Claims are largest-first, so round 0 lowers the heaviest modules concurrently, well before any 200-module batch ends. SIMPLE_MIR_FORK_BATCH was also missing from the three worker env allowlists.

Fix

  • SIMPLE_MIR_FORK_CHILD_KB sets a per-child private-memory budget. The default is 12000000 KiB; 0 turns it off.
    • Once a child holds at least one module, it reads /proc/self/smaps_rollup Private_Dirty+Private_Clean before each new claim.
    • When it is over budget, it stops claiming, codegens and publishes what it holds, and finishes. The root then forks a fresh child from its untouched post-setup heap.
  • Fork-time admission: the root forks min(requested, max(1, headroom/budget)) children, where headroom is the tightest cgroup v2 memory.max - memory.current on the chain, capped by MemAvailable. It reads files only: no subprocess or thread runs before the fork. It prints the count it chooses and the inputs. With f24's numbers it admits 4.
  • native_mir_fork_next_batch_v1 takes a batched flag (batch>0 or budget on), so the loop still terminates.
  • Allowlists: SIMPLE_MIR_FORK_BATCH and SIMPLE_MIR_FORK_CHILD_KB are added to driver_admitted_compile, native_build_environment_identity (an exclusion list, next to SIMPLE_MIR_FORK_WORKERS), and worker_hir_profile_v1, along with its spec.
  • Specs: budget parsing, the over-budget rule, smaps parsing, and admission (f24 numbers give 4; tiny headroom gives 1).
  • Design doc: section added.

Evidence

  • Independent Opus review: no P0/P1. It traced share coverage with K < N admitted children, termination, fork safety, units, cache identity and push gates.
  • Open P2 notes:
    • The budget is checked between claims, so it is soft: a single heavy module or the end-of-batch codegen can overshoot.
    • If children exceed the budget after one module, the run falls back to one module per fork.
    • memory.current includes page cache, which makes admission conservative.
  • Not executed: no full-CLI test binary exists on this host. This is effective only after the next Stage 2 rebuild, which is planned together with the dict-key regression fix.

🤖 Generated with Claude Code

Review evidence recorded in the session handoff.

p4-cli-f24 (FORK=5, batch 200, 100G cap) lost a child to the memcg OOM
killer at 52.5G anon-rss: the batch bounds claims, not bytes, and LPT
hands round 0 the heaviest modules.

- SIMPLE_MIR_FORK_CHILD_KB (default 12000000 KiB, 0 = off): a forked
  child probes /proc/self/smaps_rollup Private_Dirty+Private_Clean after
  its claim count grows; over budget (only after >=1 claim) it latches its
  cap, codegens/publishes what it holds and finishes like a full batch.
  The root's next-batch rule now keys on "batched" (batch>0 or budget on).
- Admission: the root forks min(requested, max(1, headroom/budget)),
  headroom = tightest cgroup v2 memory.max-memory.current, bounded by
  MemAvailable (direct procfs reads, no subprocess before fork).
- Admit SIMPLE_MIR_FORK_BATCH / SIMPLE_MIR_FORK_CHILD_KB next to
  SIMPLE_MIR_FORK_WORKERS in the three worker env lists.

Specs UNEXECUTED: no full-CLI simple binary is deployed on this host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ormastes
ormastes deployed to self-review-admission November 21, 2026 21:22 — with GitHub Actions Active
@ormastes
ormastes merged commit 4c9e74b into release/1.0.0 Nov 21, 2026
8 checks passed
@ormastes
ormastes deleted the work/mir-fork-child-budget branch November 21, 2026 21:24

This branch was successfully deployed

1 active deployment
self-review-admission — 35ea1868 Deployed Nov 21, 2026 by ormastes via Invalidate changed PR self-review state #41261
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant