Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
8bc4e19
test: add opt-in performance and contract reports
cpojer Nov 17, 2026
982a6ca
test: schedule Pages error contracts independently by framework
cpojer Nov 17, 2026
5c63fe4
build: cache explicit prerequisites and prepare integration artifacts…
cpojer Nov 17, 2026
b5bfd9a
test: prepare owned application fixtures with native build caching
cpojer Nov 17, 2026
d212114
test(react): share immutable island build artifacts across contracts
cpojer Nov 17, 2026
b567908
test(vue): reuse the production island fixture build
cpojer Nov 17, 2026
ac22ed0
test(svelte): cache distinct island and mixed-page builds
cpojer Nov 17, 2026
72a10d1
test(solid): share the island and private-page production build
cpojer Nov 17, 2026
0d32604
test(solid): reuse the built hydration fixture without changing dev c…
cpojer Nov 17, 2026
d37ea03
test(md): cache the standalone Markdown client-script build
cpojer Nov 17, 2026
938bd6f
test: consolidate named fixture preparation and remove anonymous buil…
cpojer Nov 17, 2026
3b56611
test: capture performance reports from affected verification
cpojer Nov 17, 2026
fc8eded
test: schedule HTTP error modes independently and reuse production ou…
cpojer Nov 17, 2026
5009505
test: run independent authentication browser contracts in separate files
cpojer Nov 17, 2026
15b856b
test: separate Cloudflare operation transport, deployment, and servic…
cpojer Nov 17, 2026
c35bcb3
perf(cli): load OAuth storage and terminal UI only for operations tha…
cpojer Nov 17, 2026
acfc740
test: use native canonical paths for fixture cache restoration
cpojer Nov 17, 2026
4d26f3e
test: prepare shared artifacts once before parallel PostgreSQL runners
cpojer Nov 17, 2026
2d24401
test(react): isolate dev mutations and preview state from authored pl…
cpojer Nov 17, 2026
9b6ff70
test: keep Vue, Svelte, and Solid runtime mutations in isolated playg…
cpojer Nov 17, 2026
5943884
test: balance native compiler concurrency and complete iteration guid…
cpojer Nov 17, 2026
aab5187
ci: share native build caching across SDK and platform checks
cpojer Nov 17, 2026
08d6004
test: preserve filename filters in the native iteration shortcut
cpojer Nov 17, 2026
58b3421
test: retain phase isolation after full-load process deadline regression
cpojer Nov 17, 2026
16a41e7
test: remove compiler limits and adapter copies that regress complete…
cpojer Nov 17, 2026
23d8469
Reduce test performance work to verified CLI startup and local toolin…
cpojer Nov 17, 2026
cc0f1a6
test: isolate SDK PGlite heaps on Windows after a native JIT crash
cpojer Nov 17, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ vp run build:core # Build shared packages, CLI, and platform r
vp run test # Affected local tests, with integration fallbacks
vp run test --list # Preview affected test files without builds/hooks
vp run test --base origin/main # Include committed branch changes
vp run test:iterate <test-file> # Native dependency-based iteration; see TESTING.md for limits
vp test run # Run tests (unit + integration tests)
vpr test:all # Format, lint, build, typecheck, and run all SDK/platform tests
vp test run packages/void # Only test the core plugin
Expand Down
32 changes: 30 additions & 2 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,21 @@ while building unrelated apps, including fixtures that have not generated their
Type-only contracts use `*.typecheck.ts`. Package typechecking verifies them; the Vitest runtime
does not need to execute their `expectTypeOf` assertions.

### Iterating on imported code

`vp run test:iterate <test-file>` uses Vitest's native dependency records to select files changed
since their last run. New tests and previously failed tests run too. The task itself is never
cached. For example:

```sh
vp run test:iterate packages/void/test/unit/plugin-routing.test.ts
```

This is an iteration shortcut. Records do not cover arbitrary filesystem reads, newly added
routes, subprocess inputs, or environment changes. For those changes, use `vp run test` with its
integration fallbacks, or run the known consumer without incremental selection. Finish with the
affected and full verification below.

### Affected local tests

`vp run test` is the local affected-tests default; `vp test run`, `vp run test:full`, and CI still
Expand Down Expand Up @@ -109,8 +124,9 @@ The cf runner's transport checks live in `cf-version-transport.test.ts` so real
not serialize the whole runner suite. Both files use the same fixture helpers with per-file
isolated roots and cleanup; keep their real HTTP, process, and deadline assertions.
Unit files and most API integration files use isolated worker threads to reduce startup time. On Linux and macOS,
API integration files and the PGlite unit files (`unit-pglite`, `platform-unit-pglite`) use forks to isolate PGlite's
WebAssembly JIT allocations after native Node 24 crashes in shared worker processes. API tests
API integration files and platform PGlite unit files (`platform-unit-pglite`) use forks to isolate PGlite's
WebAssembly JIT allocations after native Node 24 crashes in shared worker processes.
SDK PGlite units (`unit-pglite`) use forks on every platform; Windows also exhibits the JIT teardown crash. API tests
that change the process timezone also use forks, because threads cannot change Node's timezone. Select every
API file with `vp test run --project 'platform-api-integration*'`. SDK integration files use forks.
This keeps database engines from competing with all framework compilers at startup. Keep module
Expand All @@ -136,6 +152,18 @@ the cache for V8 coverage, and `NODE_V8_COVERAGE` disables it too, to preserve p

## Profiling slow tests

Capture comparable full-suite or focused measurements with the opt-in reporter:

```sh
vp run test:benchmark --outputFile=/tmp/void-tests.json
vp run test:benchmark packages/void/test/unit/plugin-routing.test.ts --outputFile=/tmp/void-routing.json
```

Reports contain total elapsed time, per-file setup/import/execution costs, test names and outcomes,
and machine/toolchain details. Every selected test executes. Compare identical selections on the
same machine and record whether builds and transform caches were prepared. Per-file durations
overlap; summing them does not measure wall-clock savings.

Vitest 5's import diagnostics are opt-in so ordinary runs stay quiet:

```sh
Expand Down
18 changes: 10 additions & 8 deletions packages/void/src/cli/cloudflare-operation-process.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,15 +20,8 @@ import {
readCfUploadOutput,
safePositional,
} from './cf-command-values.ts';
import { persistCfKeyringPreference, readCfSessionCredential } from './cf-session.ts';
import { getCfCliVersion, getVoidUserAgent } from './cf-user-agent.ts';
import { publishCloudflareWorkerSettings } from './cf-worker-settings.ts';
import {
cloudflareDeviceCode,
cloudflareVerificationUrl,
printCloudflareLoginLink,
printCloudflareLoginPrompt,
} from './cloudflare-login-output.ts';
import { platformWorkerUploadAttribution } from './platform-worker-user-agent.ts';
import { sanitizeTerminalLogText } from './terminal-log-text.ts';

Expand Down Expand Up @@ -148,7 +141,13 @@ function runCf(
return result(child.status, child.stdout ?? '', stderr);
}

function runInteractiveCf(command: Array<string>): Promise<Result> {
async function runInteractiveCf(command: Array<string>): Promise<Result> {
const {
cloudflareDeviceCode,
cloudflareVerificationUrl,
printCloudflareLoginLink,
printCloudflareLoginPrompt,
} = await import('./cloudflare-login-output.ts');
return new Promise((resolve) => {
const child = spawn(process.execPath, [cf, ...command, '--quiet'], {
cwd: process.cwd(),
Expand Down Expand Up @@ -556,6 +555,7 @@ async function sessionCredential(): Promise<
if (state.authenticated !== true || state.tokenValid !== true) {
return { failure: NO_OAUTH_BEARER };
}
const { readCfSessionCredential } = await import('./cf-session.ts');
const credential = readCfSessionCredential(process.cwd(), state);
return credential ? { type: 'oauth', token: credential } : { failure: NO_OAUTH_BEARER };
}
Expand All @@ -564,6 +564,7 @@ async function auth(): Promise<Result> {
if (operation.type === 'auth.login') {
const loggedIn = await runInteractiveCf(['auth', 'login', '--force']);
if (loggedIn.status === 0) {
const { persistCfKeyringPreference } = await import('./cf-session.ts');
persistCfKeyringPreference(process.cwd());
}
return loggedIn;
Expand All @@ -581,6 +582,7 @@ async function auth(): Promise<Result> {
return result(1, '', 'Cloudflare authentication is required.', { loggedIn: false });
}
if (String(identity.authSource).startsWith('OAuth token from ')) {
const { persistCfKeyringPreference } = await import('./cf-session.ts');
persistCfKeyringPreference(process.cwd());
}
return result(0, '', called.stderr, {
Expand Down
60 changes: 60 additions & 0 deletions packages/void/test/unit/cloudflare-operation-process.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,66 @@ import {
tokenEnv,
} from '../helpers/cloudflare-operation-process.ts';

it.each(['CLOUDFLARE_API_TOKEN', 'CF_API_TOKEN'] as const)(
'keeps OAuth storage and terminal UI unloaded when %s supplies the bearer',
async (name) => {
const root = temporaryDirectory('void-token-only-startup-');
const preload = join(root, 'reject-auth-imports.mjs');
writeFileSync(
preload,
`import { registerHooks } from 'node:module';
registerHooks({ resolve(specifier, context, nextResolve) {
if (specifier.includes('/cf-session') || specifier.includes('/cloudflare-login-output') ||
specifier.startsWith('@cloudflare/workers-auth') || specifier === '@voidzero-dev/vite-plus-prompts') {
throw new Error('Token-only operations must not load OAuth storage or terminal UI');
}
return nextResolve(specifier, context);
} });`,
);
const result = await runOperation(
root,
{ type: 'auth.token' },
tokenEnv(undefined, name),
preload,
);
expect(result.status, result.stderr).toBe(0);
expect(JSON.parse(result.stdout).data).toEqual({ type: 'api_token', token: 'synthetic-token' });
},
);

it('loads the live terminal presentation when the login operation needs it', async () => {
const root = temporaryDirectory('void-login-presentation-');
const preload = join(root, 'login-child.mjs');
writeFileSync(
preload,
`import childProcess from 'node:child_process';
import { EventEmitter } from 'node:events';
import { syncBuiltinESMExports } from 'node:module';
import { PassThrough } from 'node:stream';
childProcess.spawn = (_command, args) => {
if (!args.includes('login') || !args.includes('--force')) throw new Error('Expected forced login');
const child = new EventEmitter();
child.stdout = new PassThrough();
child.stderr = new PassThrough();
setImmediate(() => {
child.stderr.write('Visit this link to authenticate: https://example.test/device\\nand enter the code: TEST-1234\\n');
child.stdout.end(); child.stderr.end(); child.emit('close', 0, null);
});
return child;
};
syncBuiltinESMExports();`,
);
const result = await runOperation(
root,
{ type: 'auth.login' },
{ ...systemEnv, CLOUDFLARE_AUTH_USE_KEYRING: 'false' },
preload,
);
expect(result.status, result.stderr).toBe(0);
expect(result.stdout).toContain('TEST-1234');
expect(result.stdout).toContain('https://example.test/device');
});

const voidVersion = JSON.parse(readFileSync(new URL('../../package.json', import.meta.url), 'utf8'))
.version as string;
const cfVersion = JSON.parse(readFileSync(new URL(import.meta.resolve('cf/package.json')), 'utf8'))
Expand Down
27 changes: 27 additions & 0 deletions packages/void/test/unit/test-iteration.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
import { join } from 'node:path';
import { loadConfigFromFile } from 'vite-plus';
import { expect, it } from 'vite-plus/test';
import { parseCLI } from 'vite-plus/test/node';

it('keeps a forwarded filename as a filter for the native iteration command', async () => {
const loaded = await loadConfigFromFile(
{ command: 'serve', mode: 'test' },
join(import.meta.dirname, '../../../../vite.config.ts'),
);
const task = loaded?.config.run?.tasks?.['test:iterate'];
if (
!task ||
typeof task !== 'object' ||
!('command' in task) ||
typeof task.command !== 'string'
) {
throw new Error('Expected a native Vitest command');
}
const file = 'packages/void/test/unit/plugin-routing.test.ts';
const { filter, options } = parseCLI(
`${task.command.replace(/^vp test/, 'vitest')} ${file} --project unit`,
);
expect(filter).toEqual([file]);
expect(options.sinceLastRun).toBeTruthy();
expect(options.project).toEqual(['unit']);
});
73 changes: 73 additions & 0 deletions test/performance-reporter.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
import { mkdirSync, writeFileSync } from 'node:fs';
import { availableParallelism, cpus, platform, release, totalmem } from 'node:os';
import { dirname, relative, resolve } from 'node:path';
import type { Reporter, TestModule, Vitest } from 'vite-plus/test/node';

/** Opt-in measurements; this reporter never participates in test selection or result reuse. */
export default class PerformanceReporter implements Reporter {
private root = process.cwd();
private output = '';
private started = 0;

onInit(vitest: Vitest): void {
this.root = vitest.config.root;
const configured = vitest.config.outputFile;
this.output = resolve(
this.root,
(typeof configured === 'string' ? configured : configured?.performance) ??
'node_modules/.cache/void-tests/performance.json',
);
this.started = performance.now();
}

onTestRunEnd(
modules: ReadonlyArray<TestModule>,
errors: ReadonlyArray<unknown>,
reason: string,
): void {
const files = modules.map((module) => {
const diagnostic = module.diagnostic();
return {
project: module.project.name,
file: relative(this.root, module.moduleId).replaceAll('\\', '/'),
durationMs: diagnostic.duration,
collectMs: diagnostic.collectDuration,
setupMs: diagnostic.setupDuration,
prepareMs: diagnostic.prepareDuration,
environmentMs: diagnostic.environmentSetupDuration,
worker: diagnostic.concurrencyId,
tests: [...module.children.allTests()].map((test) => ({
name: test.fullName,
state: test.result().state,
durationMs: test.diagnostic()?.duration ?? 0,
})),
};
});
mkdirSync(dirname(this.output), { recursive: true });
writeFileSync(
this.output,
`${JSON.stringify(
{
version: 1,
recordedAt: new Date().toISOString(),
durationMs: performance.now() - this.started,
reason,
unhandledErrors: errors.length,
machine: {
node: process.version,
platform: platform(),
release: release(),
arch: process.arch,
cpu: cpus()[0]?.model,
parallelism: availableParallelism(),
memoryBytes: totalmem(),
},
files,
},
null,
2,
)}\n`,
);
console.log(`Test performance report: ${this.output}`);
}
}
11 changes: 8 additions & 3 deletions vite.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,11 @@ const platformAlias = {
export default defineConfig({
run: {
tasks: {
'test:iterate': { command: 'vp test run --since-last-run=true', cache: false },
'test:benchmark': {
command: 'vp test run --reporter=default --reporter=./test/performance-reporter.ts',
cache: false,
},
'build:all': {
command:
"vp run build && vp run --filter './platform/packages/**' build && vp run --filter './playground/**' --parallel build",
Expand Down Expand Up @@ -487,9 +492,9 @@ export default defineConfig({
fsModuleCache: true,
include: sdkPgliteUnitTests,
sequence: { groupOrder: 2 },
// Native Node 24 worker crashes can abort the run on Linux and macOS.
// Keep PGlite's Wasm heap in a separate process for each database.
pool: ['linux', 'darwin'].includes(process.platform) ? 'forks' : 'threads',
// PGlite's Wasm JIT teardown can crash Node 24 worker threads, including Windows.
// Keep each database in a separate process on every platform.
pool: 'forks',
maxWorkers,
...(process.platform === 'win32' ? { testTimeout: 20_000 } : {}),
},
Expand Down
Loading