Replace the runtime allocator

[SeanTAllen]

This replaces the runtime allocator on every platform with the arena allocator designed in #5735: the platform primitives, the allocator itself, cross-thread frees through per-owner batches and inboxes, the large and oversized paths, the growable owner registry, the shared-region layer the arenas are carved from, the tuning pass and its --ponymemoryprofile knob, the scheduler's suspend-and-drain integration, and the Windows port. The old pool stays in tree: use=pool_classic selects it on any platform.

The old pool can't reclaim: a large block freed on a thread that didn't allocate it is stranded for the life of the program, memory carved for one size class never serves another, and no size-class memory ever goes back to the operating system. The arena reclaims: physical memory goes back to the operating system, and address space stays parked for reuse, by design. Measured against the old pool at the earlier three-profile tuning of --ponymemoryprofile, a scheduler thread holding about 100 MiB of size-class blocks goes idle and the arena returns it: resident falls from 105 MiB to 6.5 MiB, where the old pool holds it at 104 MiB. That is the per-size-class cache #4911 asked the old pool to return, which it can't; the arena returns it on idle. The comment headed "A same-thread large-block churn regression, and a fix" has those reclaim runs and they still stand; the comment headed "Throughput and memory, remeasured" has the current throughput and memory.

The design gives every thread its own arenas, and an arena big enough to hold multi-megabyte blocks costs address space: at 128 MiB apiece, that's a 9 GiB floor on a box with 72 scheduler threads. Arenas here are 8 MiB (2 MiB on 32-bit) instead, carved out of shared 256 MiB regions (64 MiB on 32-bit) with one compare-and-swap per carve, and a region is never unmapped. Never unmapping is what keeps the lock-free carve safe — a thread walking the region list can never reach a region that is gone — and what parks an emptied arena's address space for reuse: its physical pages go back, and the next carve gets the same addresses. The cost, on Unix: reserving a region transiently maps 512 MiB, since no POSIX mapping call takes an alignment, so the reservation maps twice the size and trims the slack. Windows has no such cost. VirtualAlloc2 takes the alignment, so a region is one reservation of exactly its size.

The owner registry — where each allocator-using thread's slot and inbox live — grows a segment at a time and has no cap. An early cut of this PR used fixed arrays, which capped a process at 1,024 allocator-using threads over its lifetime and aborted the next one; the cap and the abort are gone. Slots are still never reused (nothing proves another thread holds no reference to one), and a thread that only frees never takes a slot at all: it owns nothing another thread could address.

Cross-thread frees reach their owner as runs: batches of freed objects, one run per slab, the span of arena units serving one size class. The check that every object in a run belongs to the slab it's credited to — active in debug and in a release-safe -DPONY_ALWAYS_ASSERT build, compiled out of a plain release like pony_assert — has a test seam, and forged-run death tests drive nine forge cases through it, each asserting the abort. A contended churn test overlaps the flush, drain, carve, and release paths that the single-purpose tests run one at a time; those interleavings ran under the thread sanitizer for the first time.

Suspend-and-drain closes the reclaim-timing gap: before it, memory freed for a suspended thread sat in that thread's inbox until its next allocation. There is no wake. A scheduler thread with no work goes passive and polls on a timed tick — 10 ms, doubling per quiet visit to a 500 ms cap. On every visit it drains its own allocator inbox, taking back what other threads freed to it; flushes its pending cross-thread frees so their owners can reclaim; pops the global inject queue once; and, once the tick reaches the cap, hands its idle memory back to the OS. A foreign free notifies no one: it waits in the owner's inbox — a bare atomic head pointer keyed by owner slot — and the owner reclaims it on its own next tick. A missed signal costs at most a tick, never a hang, so no lost-wake proof exists anywhere. The pinned-actor thread runs the same tick loop; the ASIO thread registers no scheduler and blocks on the OS wait, flushing its pending frees before it blocks and draining after it wakes, and that wait is bounded at 500 ms, so once a full half second passes with no event the thread returns its held memory once and then blocks unbounded until the next event. The pool unit tests exercise the cross-thread free and drain paths with real threads — DrainReclaimsDeliveredForeignFree, FreeToExitedOwner, the contended churn — and a stress program under test/rt-stress/suspend-drain drives the whole path with real schedulers: allocate on all of them, scale down, free everything from the one thread left, and the resident set drops by most of the payload while the active count holds at the floor, then climbs back when new work arrives. The program fails if the count stays raised with work outstanding, or if the memory never comes back.

The Windows port took more than reserve and commit. ponyint_virt_decommit was carrying two contracts that madvise(MADV_DONTNEED) collapses into one: the classic pool discards a freed block's pages and writes the block header back into that same range eight lines later, so the range has to stay usable, while the arena gives a unit's pages back and commits them again before reuse. The two are different calls on Windows — MEM_RESET keeps the pages committed, MEM_DECOMMIT doesn't — so one function can't serve both contracts. So it's two functions now. ponyint_virt_decommit is the arena's. ponyint_virt_discard is the classic pool's, and main's docstring for the old name described that contract all along: the range stays valid and the pages fault back in on touch.

The ASIO thread was missing the flush and drain the other backends already do. On epoll and kqueue it delivers its pending cross-thread frees before it blocks and drains after it wakes; on Windows it did neither, which cost nothing while the classic pool's flush and drain were no-ops. Flipping the default is what makes that gap matter: an idle ASIO thread would hold every foreign free it had accumulated for as long as it stayed blocked, the same reclaim gap suspend-and-drain closes everywhere else. The Windows backend flushes and drains now, placed after the WAIT_TIMEOUT check rather than straight after the wait the way the other two do it: draining calls into the OS, which overwrites the thread's last error, and losing that check strands a program reading a pipe on stdin.

VirtualAlloc2 is what makes a region's address self-aligned in one call, and it isn't in kernel32.lib. There's no narrower import library for it in the SDK, so we add onecore.lib to the link line, in libponyrt's CMake for the binaries we build and in default_libs for the programs ponyc links. Adding it doesn't move the imports we already had: ponyc.exe's import table still resolves VirtualAlloc, VirtualFree, and GetCurrentProcess from KERNEL32.dll, with only VirtualAlloc2 coming from the api-set forwarder.

The arena's 1,500 lines had never been through a C++ compiler — MSVC builds libponyrt's .c as C++, and the arena was #ifdef'd out there. Two things failed to compile: a designated initializer, now positional with a static assert pinning it, and test seams that needed PONY_EXTERN_C. The PoolArena tests and death tests ran on Windows for the first time, and one failed for real: it forges a run header into a slab whose pages the allocator has already handed back, which only ever worked because decommit was madvise everywhere. It takes the pages back with ponyint_virt_commit before forging now, which is a no-op everywhere else.

On throughput no program in the tree runs slower under the arena than under the old pool. binary-trees, which builds and holds a deep tree of small objects, runs --max-depth=21 in 12.76 s against 13.37 s under the old pool. producer-consumer, producer actors allocating 4 KiB payloads for a consumer to free and reclaim, finishes in 1.73 s under both. message-ubench, actors ping-ponging small messages, runs 39.15M msg/s against the old pool's 38.30M, or 102%. Peak resident memory, arena first and in the same order: about 20.0 GB under both for binary-trees, the arena 0.3% higher; 19.3 MB against 31.4 MB for producer-consumer; roughly 4.5 MB against 3.0 MB for message-ubench. Measured on a 16-core box, five interleaved rounds with the high and low dropped and the middle three averaged, each program timed by its own clock, and no run given more scheduler threads than the program has busy actors. A per-class thread cache fronts the slab path, and it takes every freed small block whatever thread owns it: a block freed on the wrong thread lands in the freeing thread's cache and gets handed straight back out from there, and that is what took the cross-thread routing cost out of message-ubench's number. A block only gets an owner assigned when it moves toward the slabs or its owner's chain — when it arrives at a cache that is already full, on the idle return, or at thread teardown — so a foreign free sitting in the cache is not stranded the way it is under the old pool, where it stays on the freeing thread's list for the life of the program.

One shape trails: a working set the thread cache is not deep enough to hold. Depth per size class is the larger of two numbers, a byte budget divided by the class size and a floor count of blocks kept regardless of the budget, hard-capped at 512 blocks. --ponymemoryprofile is ten rungs and both numbers are columns in it — the budget is 64 KiB at rung 1, 640 KiB at the default rung 3, and 2 MiB at rungs 9 and 10, and the floor is 0 at rung 1, 8 at the default, and 128 at rung 10. The floor is there because the budget on its own rounds down to a handful of blocks or none at all for the largest classes, and without it a thread churning a large working set pays a slab reserve and a slab release every cycle.

One more of the dial's columns is the large-retention budget, the bytes of freed large-block and oversized memory a thread may keep committed: 0 at rung 1, 16 MiB at the default rung 3, and 128 MiB at rung 10. A freed large block's span keeps its pages: freed pages sit committed until a sweep returns them, and a kept span is marked so the sweeps that batch small-slab returns leave it alone, while any later carve of any size consumes the warm pages. A freed oversized mapping is kept whole by the thread that freed it, keyed by its power-of-two reservation, the quantity that repeats across same-size cycles and realloc growth chains. The budget bounds both by admission: a free that does not fit is released to the operating system at once, and no allocation evicts what is retained. The one exception: if the oldest kept mapping sits under a different reservation than an incoming free, and evicting it would let that free fit, the old one is unmapped and the new one kept — one unmap either way.

Measured on the same box, three timing rounds with order reversed, against this branch with retention off: a mixed-churn ring churning 12 MiB frames runs 5.3x faster once the budget covers the circulating mappings, rung 5 of the dial; realloc growth chains that cross the boundary between the large and oversized tiers run 4.4x faster at budgets that cover them; actor-churn's destruction churn gains 6-11% at rungs 3 and 5; and the cross-thread ring of large blocks gains up to 54% at the deepest budgets. The actor heap's chunk recycler already absorbs same-actor rebuild churn before it reaches the pool, so the win lives in the shapes the recycler cannot serve: actor destruction, growth chains, oversized cycles, cross-thread traffic. And the bound has a price: at rung 10 an early cut of this PR kept each arena's freed large-block pages with no bound at all, and replacing that with the declared bound costs the destruction shape 34% against that unbounded accident.

mixed-churn, a benchmark under benchmark/memory-profile that varies a churn cycle around what the cache holds, runs a ring of eight workers, each one freeing an incoming burst of another thread's 4 KiB blocks and keeping scratch blocks of its own resident. Its size cap is now 64 MiB, so its churn reaches the large and oversized tiers as well. At 16-block bursts and 8 scratch it runs 3.46M batches/s against the old pool's 3.37M, on 13 MB against 18 MB; a cycle that stays inside the cache never reaches the slabs or the decommit machinery, so the dial's memory-return settings make no difference to that traffic. At 128-block bursts and 32 scratch the cycle is 160 blocks, exactly what 640 KiB of 4 KiB blocks comes to at the default: 800k batches/s against 824k, or 97%, on 13.9 MB against 17.4 MB. At 512-block bursts and 64 scratch the cycle is 576, past the hard cap of 512, and the cache is not deep enough to hold it at any rung: 41k batches/s at the deepest cap against the old pool's 62k, the old pool 1.5x ahead.

The old pool is faster there for the same reason it can't reclaim: freed blocks stay on its free lists for the life of the program, so a working set past any depth the cache can reach is one it serves from a push and a pop. The arena's cache is bounded on purpose, and so is the retention. A thread in the middle of churning holds per size class at most the larger of the byte budget and the floor, plus the large-retention budget across the large and oversized tiers, 16 MiB at the default, and the idle return flushes the cache, the retained spans, and the kept mappings, so a parked thread holds nothing. Reclaim is delayed by what a thread is holding, never lost: every pausing runtime thread hands everything back once it has been quiet long enough. That is the price of boundedness.

The arena carries no AddressSanitizer, Valgrind, or pooltrack instrumentation, and a build that combines it with address_sanitizer, valgrind, or pooltrack stops with an error saying to pair the tool with pool_classic (or pool_memalign, for ASan) — a clean run that checked nothing would mislead. The FreeBSD valgrind smoke builds pool_classic,valgrind, the weekly pool_retain row becomes pool_classic,pool_retain, and new pool_classic rows in the weekly matrices build and test the old pool. Every other Unix job that doesn't name a pool option builds the arena default.

Verified locally, beyond what PR CI runs: ci-core is green under use=pool_classic; the classic-retain and memalign suites are green; a bare use=address_sanitizer build fails with the pairing message; a ThreadSanitizer run over the allocator tests, the contended churn and the inbox-drain tests included, came back clean. On Windows the runtime unit suite passes, pool_classic still builds, pool_retain and pooltrack each build paired with it and each fail alone with the #error naming the fix, and reordering pool_arena_thread_t fails the build on the static assert that pins its positional initializer. I proved every new test but one able to fail by breaking the code under it; the exception pins a no-crash property, a foreign free to an owner that has already exited.

Rewriting the realloc path turned up a latent bug present in all three backends (the arena, the classic pool, and memalign): shrinking an allocation from above POOL_MAX to a size-class size freed the old block with size 0. No current caller reaches the path. The fix rides in the arena commit, and the realloc contract test now covers the path under every backend. I also narrowed one pre-existing test: Heap.Init probed the pagemap one past a large chunk and asserted the entry belonged to neither the chunk nor its actor; under the arena's dense placement a neighboring chunk of the same heap can legitimately be there, so the probe now asserts only that the entry isn't the chunk itself.

The allocator's internal design is documented in src/libponyrt/mem/POOL_ARENA.md.

Design: #5735