Add split buffer facility to support QNX PMEM and other buffering systems#72
Merged
Conversation
Gate QNX persistent-memory buffers behind build-time flags and register metadata with the server so channel teardown can clean up pmem objects.
Track QNX PMEM buffer metadata through client registration, server cleanup, and shadow replication so servers can recover and release client-allocated PMEM safely.
Bring in the merged CMake build fixes and resolve the plugin/test wiring against the QNX PMEM branch.
Expose custom PMEM allocation and mapping hooks so embedders can manage QNX PMEM buffers while preserving metadata needed for subscribers.
Simplify split-buffer setup so allocator-specific behavior is supplied by callbacks instead of protocol and option fields. Also record Bazelisk as the repository test runner for future agent work.
Allow test binaries to opt into split-buffer publishers by default and fix split-buffer multi-publisher attachment and prefix handling so client_test can run under that mode.
Keep the default test dimensions unchanged while reducing split-buffer-only sweeps that exceed common per-process mapping limits.
dallison
changed the title
Add explicit CI coverage for client_test, latency_test, and stress_test with split-buffer publishers enabled.
Avoid macOS file descriptor exhaustion when the split-buffer client test maps many publisher slot metadata files in one process.
Keep Linux split-buffer coverage unchanged while lowering macOS-only latency and stress shapes that exceed the runner file descriptor limit.
Reduce split-buffer latency message counts and further lower macOS mux dimensions so the new CI coverage fits runner limits.
Optimized Latency ReportParsed 2116 latency records from optimized latency runs.
ChartsSVG charts and raw JSONL are attached to the workflow run. The chart bundle is available as the The old major-regression comparison table was removed because CI-hosted baseline comparisons were too noisy and no longer reflect the current CI policy. Current CI keeps the optimized latency smoke run, but does not publish or gate on the major-regression table. |
Keep the latency report tooling available for manual regression runs while avoiding noisy cloud benchmark comparisons in GitHub Actions.
Run the optimized latency test in CI without baseline comparison or PR regression reporting, keeping the check useful without noisy cloud deltas.
Add C API accessors for publisher/subscriber slot addresses and subscriber timeout waits so external clients can inspect mapped buffers directly.
Adds missing C client API wrappers and focused tests, including split-buffer, checksum callback, ASAN, and Valgrind CI coverage.
Mirrors generic split-buffer metadata through the shadow process and removes the obsolete PMEM-specific build surface.
Avoids a concurrent publisher attach race where mmap can see an object before the creator has finished sizing it.
Allows the lossy callback test to tolerate extra legitimate drops observed under macOS scheduling.
Avoids replacing shadow channel FDs during option updates and gives split-buffer tests unique shared-memory names under parallel runs.
Use mkstemp-backed socket paths for Rust server fixtures so repeated Bazel runs do not collide across parallel sandboxes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--use_split_buffers.Test plan
bazelisk test //...bazelisk test //client:client_test //client:latency_test //client:stress_test --test_arg=--use_split_buffersbazelisk test --config=gcc //...bazelisk test --config=clang //...bazelisk test --config=asan //client:client_test --test_arg=--use_split_buffersbazelisk test //client:client_test --run_under='valgrind --error-exitcode=1 --errors-for-leak-kinds=definite' --test_timeout=600bazelisk test //c_client:client_testbazelisk test //c_client:client_test --test_arg=--use_split_buffersbazelisk test --config=asan //c_client:client_testvalgrind --tool=memcheck --leak-check=full --show-leak-kinds=definite,indirect --errors-for-leak-kinds=definite,indirect --error-exitcode=99 bazel-bin/c_client/client_testcargo testin the Rust crates touched by this PRcmake -S . -B build/cmake -DCMAKE_BUILD_TYPE=Debugcmake --build build/cmake --parallel $(nproc)ctest --test-dir build/cmake --output-on-failureNotes