test(e2e): cover dashboard port auto-alloc in nightly

[cjagwani]

Summary

Adds three #2174 regression assertions to the existing test-double-onboard.sh and wires it into nightly-e2e.yaml as a new double-onboard-e2e job. Without these, a regression of the auto-alloc path (crashing second onboards or silently stealing the first sandbox's forward) would ship to main with only unit tests as a guard.

Related

Depends on #2454 (auto-alloc behavior must be on main before the assertions can pass).

Changes

• test/e2e/test-double-onboard.sh — three new assertions added to the existing Phase 4 (second-sandbox onboard):
  • Second onboard's stdout must include allocated port (confirms auto-alloc fired)
  • nemoclaw list must show two distinct dashboard: http://... lines for the two sandboxes
  • Existing "first sandbox still alive" + "port 18789 conflict not detected" assertions preserved
• .github/workflows/nightly-e2e.yaml — new double-onboard-e2e job that runs the script. Uses the script's built-in fake OpenAI-compatible endpoint so no NVIDIA_API_KEY is needed. Wired into notify-on-failure needs: list so a regression auto-opens a GitHub issue.

Type of Change

• Code change (feature, bug fix, or refactor)

Verification

• npx prek run --all-files passes (YAML validates, shellcheck clean)
• bash -n test/e2e/test-double-onboard.sh — syntax check passes
• Script runs locally against a live sandbox pair once fix(onboard): auto-allocate dashboard port on multi-sandbox conflict #2454 lands
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed

Notes for reviewer

• This is the first multi-sandbox E2E in nightly. Pattern is copy-ready for future dual-sandbox scenarios ([Sandbox][Recovery] Pod restart leaves OpenClaw gateway and dashboard port-forward dead; recovery is a side-effect of nemoclaw connect #2042, [Brev Launchable] Openclaw UI link is unhealthy #1178).
• The new job's if: gate matches existing jobs (github.repository == 'NVIDIA/NemoClaw') so forks don't pay the runner cost.
• Split from fix(onboard): auto-allocate dashboard port on multi-sandbox conflict #2454 to keep that PR focused on the minimum [All Platform][Onboard][Regression] Second onboard crashes with unhandled exception and leaks ghost sandbox when dashboard port 18789 is already forwarded #2174 close.

AI Disclosure

• AI-assisted — tool: Claude Code

---

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Summary by CodeRabbit

• Tests

  • Added validation for concurrent sandbox onboarding with dashboard port collision detection.

• Chores

  • Enhanced nightly automated testing with improved failure tracking and artifact collection.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7fca9582-2bbc-4ccc-b7d0-5361487290c1

📥 Commits

Reviewing files that changed from the base of the PR and between 260b237 and 65b9b36.

📒 Files selected for processing (2)

• .github/workflows/nightly-e2e.yaml
• test/e2e/test-double-onboard.sh

---

📝 Walkthrough

Walkthrough

A new nightly E2E test job is added to validate concurrent sandbox onboarding without dashboard port collisions, integrated into the existing CI failure notification mechanism using a dedicated test script and local fake OpenAI-compatible endpoint.

Changes

Cohort / File(s)	Summary
CI Workflow Enhancement .github/workflows/nightly-e2e.yaml	New double-onboard-e2e GitHub Actions job that executes concurrent onboarding test, produces artifact logs on failure, and connects to existing failure notification pipeline via needs dependency.
Test Implementation test/e2e/test-double-onboard.sh	New bash test script that validates two sandboxes onboard simultaneously with distinct dashboard ports, asserting allocation messages and deduplicating port extraction from nemoclaw list output.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

🐰 Two sandboxes hop and play,
Each claiming ports their own way,
Concurrent dance, no collisions here,
The ports stay unique, crystal clear! 🎉

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'test(e2e): cover dashboard port auto-alloc in nightly' accurately describes the main change: adding E2E test coverage for dashboard port auto-allocation in the nightly workflow.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch test/e2e-double-onboard-port-alloc

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}