Skip to content

net: remove unnecessary locking from RX buffer pools#3152

Merged
jstarks merged 2 commits intomicrosoft:mainfrom
jstarks:queue-faster
Mar 30, 2026
Merged

net: remove unnecessary locking from RX buffer pools#3152
jstarks merged 2 commits intomicrosoft:mainfrom
jstarks:queue-faster

Conversation

@jstarks
Copy link
Copy Markdown
Member

@jstarks jstarks commented Mar 28, 2026

The RX buffer pools in virtio-net and GDMA/BNIC were wrapped in Arc<Mutex<>> (or Arc<Vec<Mutex<>>>), but they are only ever accessed from the single task that runs the queue. The locking was vestigial from an earlier design where the pool was shared. Removing it simplifies the code--no more Arc cloning, scratch buffers, or lock() calls on every buffer operation.

Performance is basically unchanged by this, which is not too surprising since the locks are uncontended and there is so much other lower-hanging fruit. But this change simplifies the code, so we should go ahead and take it anyway.

The RX buffer pools in virtio-net and GDMA/BNIC were wrapped in
Arc<Mutex<>> (or Arc<Vec<Mutex<>>>), but they are only ever accessed
from the single task that runs the queue. The locking was vestigial from
an earlier design where the pool was shared. Removing it simplifies the
code--no more Arc cloning, scratch buffers, or lock() calls on every
buffer operation.

Only TX throughput improved measurably with this change, and only by a
couple percent, but the code is simpler and easier to reason about,
so we'll take it.
@jstarks jstarks requested a review from a team as a code owner March 28, 2026 22:34
Copilot AI review requested due to automatic review settings March 28, 2026 22:34
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Removes vestigial locking/Arc indirection around RX buffer pools in virtio-net and GDMA/BNIC by making the pools task-local, simplifying buffer management and reducing per-packet overhead.

Changes:

  • virtio-net: removes ActiveState.mem and uses the RX pool’s GuestMemory; replaces “collect ready IDs” allocation with an in-place fill_ready call.
  • virtio-net buffers: replaces Arc<Vec<Mutex<...>>> with Vec<Option<...>> and adjusts APIs to require &mut self where mutation occurs.
  • GDMA/BNIC: replaces Arc<Mutex<Slab<_>>> + scratch cloning with a task-local Slab<_> stored in GuestBuffers.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
vm/devices/virtio/virtio_net/src/lib.rs Switches queue restart + TX header reads to use the RX pool as the single owner of GuestMemory and removes clone/lock-based ready collection.
vm/devices/virtio/virtio_net/src/buffers.rs Makes the virtio RX work pool single-owner (no Arc<Mutex<...>>), adds fill_ready and exposes mem() accessor.
vm/devices/net/gdma/src/bnic.rs Makes BNIC RX packet storage task-local (no Arc<Mutex<_>>) and removes scratch segment cloning.

benhillis
benhillis previously approved these changes Mar 30, 2026
@github-actions
Copy link
Copy Markdown

@jstarks jstarks merged commit d3cf566 into microsoft:main Mar 30, 2026
80 of 82 checks passed
@jstarks jstarks deleted the queue-faster branch March 30, 2026 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants