Skip to content

feat: add security load test and regression tooling#462

Merged
cameri merged 8 commits intocameri:mainfrom
YashIIT0909:poc-zombie-leak
Apr 18, 2026
Merged

feat: add security load test and regression tooling#462
cameri merged 8 commits intocameri:mainfrom
YashIIT0909:poc-zombie-leak

Conversation

@YashIIT0909
Copy link
Copy Markdown
Collaborator

Description

This PR introduces a specialized security and load-testing tool, scripts/security-load-test.js. These assets are designed to reproduce, verify, and permanently safeguard the relay against memory leaks in the WebSocket heartbeat mechanism (specifically the "zombie connection" issue).

The tool has been integrated into package.json via the npm run test:load command to allow for easy execution during manual security audits.

Related Issue

Fixes #429

Motivation and Context

While the primary heartbeat bug (eviction blocked by active subscriptions) has been addressed, the relay previously lacked a reliable way to verify that dead connections are actually being evicted from both the ws client set and the internal WeakMap adapters.

This change is required to:

  1. Prevent Regressions: Ensure future refactors of the adapter logic do not accidentally re-introduce connection-holding leaks.
  2. Security Auditing: Provide a tool to simulate Slowloris-style attacks where clients remain silently connected to exhaust file descriptors and memory.
  3. Observability: Add instrumentation logs (via heartbeat cycles) to monitor real-time object retention.

How Has This Been Tested?

The tooling was tested in a Linux environment using a 1-worker cluster configuration:

  1. Scenario 1 (Vulnerable State): Verified that 5,000 "zombie" connections (ignoring pongs with active subscriptions) were correctly identified as leaked via Heap Comparison snapshots (+5000 WebSocketAdapter delta).
  2. Scenario 2 (Fixed State): Verified that the same 5,000 connections were successfully evicted by the server after the heartbeat interval, returning the heap to baseline levels (~20MB).
  3. Scenario 3 (Combined Load): Verified the server's resilience when 5,000 zombies were present alongside active event spamming (100 events/sec).

Screenshots

Leak Evidence Description
Heap Delta Snapshot Comparison: Shows exactly +5,000 WebSocket and WebSocketAdapter objects retained in the heap. The Retainers panel confirms the objects are pinned by the WeakMap while the socket remains in the server's active client set.
Terminal Proof Script Execution: The PoC script establishing 5,000 zombies and confirming they remain silent with pong frames suppressed.
Heartbeat Logs Real-time Instrumentation: Server-side logs showing the accumulation of 5,000 ghost clients and the corresponding increase in heap usage.

Types of changes

  • Non-functional change (docs, style, minor refactor)
  • New feature (non-breaking change which adds functionality)

Checklist:

  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my code changes.
  • All new and existing tests passed.

Copy link
Copy Markdown
Owner

@cameri cameri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add documentation on how to use npm run test:load

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new manual security/load-testing utility to reproduce and guard against WebSocket “zombie connection” heartbeat regressions, and wires it into the repo’s npm scripts.

Changes:

  • Added scripts/security-load-test.js to generate zombie WebSocket connections and optionally flood the relay with valid signed Nostr events.
  • Added npm run test:load entry in package.json to run the new tooling.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
scripts/security-load-test.js New load/security test script to simulate zombie clients + optional event spammer.
package.json Adds test:load script to execute the new tool.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread scripts/security-load-test.js Outdated
Comment on lines +24 to +31
const args = process.argv.slice(2).reduce((acc, arg, i, arr) => {
if (arg.startsWith('--')) acc[arg.slice(2)] = arr[i + 1];
return acc;
}, {});

const RELAY_URL = args.url || 'ws://localhost:8008';
const TOTAL_ZOMBIES = parseInt(args.zombies || '5000', 10);
const SPAM_RATE = parseInt(args['spam-rate'] || '0', 10);
Copy link

Copilot AI Apr 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CLI parsing logic treats every --flag as having a value at arr[i + 1] (even if the next token is another flag), which can lead to TOTAL_ZOMBIES / SPAM_RATE becoming NaN (e.g., --zombies --spam-rate 100). Consider using a more robust parser (or at least validating parseInt(...) results and exiting with a helpful message when values are missing/invalid).

Suggested change
const args = process.argv.slice(2).reduce((acc, arg, i, arr) => {
if (arg.startsWith('--')) acc[arg.slice(2)] = arr[i + 1];
return acc;
}, {});
const RELAY_URL = args.url || 'ws://localhost:8008';
const TOTAL_ZOMBIES = parseInt(args.zombies || '5000', 10);
const SPAM_RATE = parseInt(args['spam-rate'] || '0', 10);
function parseCliArgs(argv) {
const acc = {};
for (let i = 0; i < argv.length; i++) {
const arg = argv[i];
if (!arg.startsWith('--')) continue;
const key = arg.slice(2);
const value = argv[i + 1];
if (value === undefined || value.startsWith('--')) {
console.error(`Missing value for --${key}`);
process.exit(1);
}
acc[key] = value;
i++;
}
return acc;
}
function parseIntegerArg(value, defaultValue, flagName) {
if (value === undefined) return defaultValue;
const parsed = Number.parseInt(value, 10);
if (Number.isNaN(parsed)) {
console.error(`Invalid value for --${flagName}: ${value}. Expected an integer.`);
process.exit(1);
}
return parsed;
}
const args = parseCliArgs(process.argv.slice(2));
const RELAY_URL = args.url || 'ws://localhost:8008';
const TOTAL_ZOMBIES = parseIntegerArg(args.zombies, 5000, 'zombies');
const SPAM_RATE = parseIntegerArg(args['spam-rate'], 0, 'spam-rate');

Copilot uses AI. Check for mistakes.
Comment thread scripts/security-load-test.js Outdated
Comment thread scripts/security-load-test.js Outdated
Comment thread scripts/security-load-test.js Outdated
Comment thread scripts/security-load-test.js Outdated
Comment thread package.json
@YashIIT0909 YashIIT0909 requested a review from cameri April 11, 2026 14:56
Copy link
Copy Markdown
Owner

@cameri cameri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's stick to TypeScript.

Comment thread scripts/security-load-test.js Outdated
@YashIIT0909 YashIIT0909 requested a review from cameri April 18, 2026 11:52
@YashIIT0909
Copy link
Copy Markdown
Collaborator Author

I have migrated the script to typescript.

@cameri
Copy link
Copy Markdown
Owner

cameri commented Apr 18, 2026

@YashIIT0909 you will have to reset your branch to your last own commit, then merge latest origin/main back into it.
There was a bad commit in main which you pulled in that does not follow commitlinf.

@YashIIT0909 YashIIT0909 force-pushed the poc-zombie-leak branch 3 times, most recently from c3182c3 to 8d2c6f9 Compare April 18, 2026 14:37
@cameri cameri merged commit 27d8f8a into cameri:main Apr 18, 2026
9 checks passed
@YashIIT0909 YashIIT0909 deleted the poc-zombie-leak branch April 18, 2026 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Critical DoS: Zombie connections with active subscriptions bypass WebSocket heartbeat timeout

3 participants