Repository navigation
benchsdk: clean up SDK surface, remove admin API key, validate CLI flags, and sanitize scaffold names - #362
Merged
Merged
Conversation
…caffolding Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor License AgreementAll contributors are covered by a CLA. |
…flag rejection Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ndencies Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Contributor
Browser Benchmark Results
View full run · SVG available as build artifact |
Contributor
Storage Benchmark Results1MB Files
View full run · SVGs available as build artifacts |
Contributor
Snapshot/Fork Benchmark Resultssmall dataset
|
Contributor
Sandbox Benchmark ResultsBurst
View full run · SVGs available as build artifacts |
Contributor
Browser Throughput Benchmark Results
View full run · SVG available as build artifact |
Contributor
Sandbox Dax Benchmark Results
|
Contributor
AI Gateway Benchmark Results — OpenAI
View full run · SVG available as a build artifact |
Contributor
AI Gateway Benchmark Results — Anthropic
View full run · SVG available as a build artifact |
Contributor
AI Gateway Benchmark Results — Gemini
View full run · SVG available as a build artifact |
Contributor
AI Gateway Benchmark Results — Kimi
View full run · SVG available as a build artifact |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tightens the
benchsdkpackages so the SDK surface is single-sourced, auth is predictable, CLI/scaffold behavior is safer, and CI only requests the benchmarks platform API key.Changes
Single-source
@benchsdk/clientfrom@benchsdk/api+@benchsdk/worker.packages/benchsdk/src/types.tsand the thin re-export files (metrics.ts,participants.ts,reporter.ts).index.tsnow re-exports the public API/worker surface, andclient.tsextendsBenchmarkClientfrom@benchsdk/apiwithrunWorker.Eliminate unsafe casts in
client.ts.runWorkeris now typed as:createBenchmarkClientcallsrunWorker(apiClient, options)directly withoutas any.Stop library code from exiting the process.
selectParticipantsin@benchsdk/workernow throws a descriptiveErrorinstead of callingprocess.exit(1).Remove
COMPUTESDK_ADMIN_API_KEYfrom all auth paths.@benchsdk/apinow checksBENCHMARKS_PLATFORM_API_KEY → COMPUTESDK_API_KEY.@benchsdk/runnerresolvePlatformnow checksBENCHMARKS_PLATFORM_API_KEY → COMPUTESDK_API_KEY.process.env.BENCHMARKS_PLATFORM_API_KEY ?? process.env.COMPUTESDK_API_KEY.COMPUTESDK_ADMIN_API_KEYfrom all benchmark workflow vault-secret allowlists.Reject unknown CLI flags in
@benchsdk/runner.parseCliArgscollects unrecognized flags and throwsUnknown flag(s): ...unless the flag is declared inBenchmarkConfig.customCliFlags.--file-size,--dataset,--ai-gateway-iterations-cold,--ai-gateway-iterations-warm).Sanitize
create-benchproject names.-, invalid characters stripped, and runs of-collapsed sopackage.jsonnameis always valid.Fix empty-package test scripts.
@benchsdk/apiand@benchsdk/workernow usevitest run --passWithNoTests, sopnpm testno longer fails when no test files exist.Fix scale image bundle.
bundle:scalenow emits ESM (dist/scale.mjs) and injects acreateRequireshim so bundled CJS dependencies such asdotenvcan still require Node built-ins at runtime.benchmarks/scale/Dockerfileandbenchmarks/scale/scripts/start.tsto run/app/coordinator.mjs.Resolve TypeScript/
@types/nodedrift.devDependencieswith the workspace packages (typescript ^5.0.0,@types/node ^20.0.0).Verification
pnpm typecheckpassedpnpm -r --filter "./packages/*" buildpassedpnpm -r --filter "./packages/*" testpassedpnpm run bundle:scalepassedLink to Devin session: https://app.devin.ai/sessions/6f1c1763ac8b47a49203e9749b5b88cd
Requested by: @dtice25