Skip to content

feat: add hub-managed port pool for agent containers#298

Open
zeroasterisk wants to merge 3 commits into
GoogleCloudPlatform:mainfrom
zeroasterisk:feature/port-assignment
Open

feat: add hub-managed port pool for agent containers#298
zeroasterisk wants to merge 3 commits into
GoogleCloudPlatform:mainfrom
zeroasterisk:feature/port-assignment

Conversation

@zeroasterisk

@zeroasterisk zeroasterisk commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Problem

Agents that need to run local web servers (dev servers, preview apps, browser-based verification) have no way to get unique, non-colliding host ports. With multiple agents on the same broker, exported ports collide. Agents also can't tell users a URL to reach a locally running service.

Solution

A broker-managed port pool that allocates unique host ports per agent at container creation time, exposed as environment variables:

  • AVAILABLE_LOCALHOST_PORT_A, AVAILABLE_LOCALHOST_PORT_B — raw port numbers
  • AVAILABLE_LOCALHOST_URL_A, AVAILABLE_LOCALHOST_URL_B — full URLs (when host_url is configured)

Ports are published via docker -p and released on stop/delete/start-failure.

Configuration

server:
  broker:
    port_pool:
      enabled: true
      range: "8000-9000"
      ports_per_agent: 2
      host_url: "http://35.232.118.211"  # optional, enables URL env vars

Reviewability roadmap

Start with pkg/runtime/portpool.go — the thread-safe allocator (~150 lines, sync.Mutex, lowest-available selection). Everything else is wiring and tests:

  1. pkg/runtime/portpool.go + portpool_test.go — core allocator + table-driven tests (50-goroutine concurrency test)
  2. pkg/runtime/common.go — env var injection + -p port publishing in buildCommonRunArgs()
  3. pkg/agent/run.go / manager.go — allocation on start, release on stop/delete
  4. pkg/api/types.go, pkg/config/settings_v1.go — config structs
  5. cmd/server_foreground.go — pool init from broker settings

Non-goals

  • No port persistence across broker restarts (in-memory only — ports reclaimed on restart)
  • No dynamic port requests from agents at runtime — allocation is at container creation only

Risk

  • No behavior change for existing agents: the pool is nil unless port_pool is explicitly configured in broker settings. Zero impact on agents that don't use port config.
  • All runtimes supported: Docker, Podman, Apple Container, Kubernetes

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a hub-managed port pool mechanism to allocate unique host ports for agent containers. It adds configuration options, implements the PortPool logic with concurrency support and tests, and integrates port allocation and release into the agent manager's lifecycle (start, stop, delete). It also injects these allocated ports as environment variables and publishes them in both Docker and Kubernetes runtimes. Feedback on the PR highlights two issues: first, recreating or restarting an agent can bypass the manager's Delete method, leading to leaked ports and eventual pool exhaustion, so releasing existing ports before allocation is suggested; second, if the host URL is configured with a trailing slash, the constructed agent port URLs will be invalid, so trimming trailing slashes during initialization is recommended.

Comment thread pkg/agent/run.go
Comment on lines +821 to +827
if m.PortPool != nil {
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

When an agent container is recreated or restarted, the existing container is deleted via m.Runtime.Delete directly, which bypasses the manager's Delete method and does not release the allocated ports back to the pool. Consequently, calling Allocate again for the same agent name will leak the previously allocated ports, eventually leading to port pool exhaustion.

To prevent this resource leak, release any existing ports for the agent name from the pool before allocating new ones.

Suggested change
if m.PortPool != nil {
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}
if m.PortPool != nil {
m.PortPool.Release(opts.Name)
ports, err := m.PortPool.Allocate(opts.Name, m.PortPool.PerAgent())
if err != nil {
return nil, fmt.Errorf("port allocation failed: %w", err)
}
allocatedPorts = ports
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed — Release is called before Allocate in the Start path to handle the restart case where old ports were not freed.

Comment thread cmd/server_foreground.go
Comment on lines +1091 to +1094
} else {
mgr.PortPool = runtime.NewPortPool(pMin, pMax, perAgent, pp.HostURL)
log.Printf("Port pool initialized: range %d-%d, %d ports per agent, host_url=%q", pMin, pMax, perAgent, pp.HostURL)
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

If pp.HostURL is configured with a trailing slash (e.g., http://35.232.118.211/), the constructed agent port URLs will contain an invalid double-slash/colon sequence (e.g., http://35.232.118.211/:8000).

Trimming any trailing slashes from the host URL during initialization ensures that the constructed URLs are always valid.

			} else {
				hostURL := strings.TrimRight(pp.HostURL, "/")
				mgr.PortPool = runtime.NewPortPool(pMin, pMax, perAgent, hostURL)
				log.Printf("Port pool initialized: range %d-%d, %d ports per agent, host_url=%q", pMin, pMax, perAgent, hostURL)
			}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Already addressed — strings.TrimRight(pp.HostURL, "/") is applied when creating the PortPool in server_foreground.go (see commit 0da4a06).

@zeroasterisk zeroasterisk force-pushed the feature/port-assignment branch from 2cb45e9 to a0d14ea Compare June 7, 2026 22:55
@zeroasterisk

Copy link
Copy Markdown
Contributor Author

3 review rounds complete, all clean. Summary of all changes since initial PR:

  • Fixed port leak on agent restart (Release before Allocate)
  • Fixed trailing slash on host_url producing invalid URLs
  • NewPortPool now validates inputs and returns error
  • Allocate validates agent name and count
  • Added ParsePortRange unit tests
  • Rebased on upstream/main (resolved conflict with NFS workspace fields)
  • All tests pass

Agents that run local web servers (dev servers, preview apps, test
harnesses) need guaranteed unique host ports. This adds a port pool
managed by the runtime broker that allocates unique ports from a
configurable range at agent startup and releases them on stop/delete.

Each agent receives environment variables:
- AVAILABLE_LOCALHOST_PORT_A, _B (port numbers)
- AVAILABLE_LOCALHOST_URL_A, _B (full URLs, when host_url is configured)

Ports are published via docker -p so they're reachable from the host.

Configuration in settings.yaml:
  server:
    broker:
      port_pool:
        range: "8000-9000"
        ports_per_agent: 2
        host_url: "http://broker-host-ip"

Defaults: range 8000-9000, 2 ports per agent, disabled unless configured.
Fixes from code review:

1. Port leak on agent restart: when an agent is stopped and restarted,
   Start() allocated new ports without releasing the old ones (which
   weren't freed because the previous run didn't go through Delete).
   Fix: call Release(name) before Allocate — idempotent no-op if no
   prior allocation exists.

2. Invalid URL with trailing slash: if host_url was configured as
   "http://example.com/", the constructed URL became
   "http://example.com/:8042". Fix: trim trailing slashes on init.
… review

Debug/refactor findings:
- NewPortPool now returns error on invalid inputs (bounds, min>max, perAgent<=0)
- Allocate validates agent name and count
- Added ParsePortRange unit tests
- Added validation error tests for PortPool constructor
- Updated callers to handle new error return
- 267 lines of improvements, all tests pass
@zeroasterisk zeroasterisk force-pushed the feature/port-assignment branch from 3903982 to b1d8e19 Compare June 27, 2026 15:19
@ptone

ptone commented Jul 2, 2026

Copy link
Copy Markdown
Member

given agents may be running in k8s or soon even more managed container platforms the "host port" may not even be a thing. as such i'm favoring a design which would have a UX a bit more like how VS Code on remote machines , or cloud shell handles exposing workload ports via port forwarding over ssh

see ptone#303

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants