Skip to content

feat(telemetry): opt-in public profile and personal usage rank - #8393

Draft
lidge-jun wants to merge 143 commits into
devfrom
goat/telemetry-07-personal-rank
Draft

lidge-jun wants to merge 143 commits into
devfrom
goat/telemetry-07-personal-rank

Conversation

@lidge-jun

@lidge-jun lidge-jun commented Nov 8, 2026 •

Copy link
Copy Markdown
Owner

Builds on the telemetry stack now on dev (#8300, #8348, #8303, #8304, #8305, #8308).

Closes #8372

Summary

People who keep anonymous telemetry on can now choose to appear on a public usage leaderboard and see their own rank. The design follows the tokscale leaderboard (explicit join, dry-run preview, server-side checks, full deletion), but publishes far less: no devices, session timing, MCP names or e-mail.

  • Separate, human-only consent. A new Public profile & rank card on the dashboard Home (shown while telemetry is on, or while a profile exists) and ocx telemetry profile connect|disconnect|status. Connecting needs a human dashboard session and an unchecked-by-default "this is public" acknowledgement, or an interactive terminal with a default-no prompt; agents get agent_consent_required/tty_required. Connecting never touches the telemetry decision, and turning telemetry off only pauses sending.
  • Identity: a chosen handle plus a server-issued secret. The secret lives in telemetry-profile.json (0600) and is never shown; the collector stores only its SHA-256. GitHub sign-in needs a maintainer-created OAuth app and is left as a follow-up.
  • Aggregates only. Once per completed UTC day after the connect day: request count, five token buckets and per-model-family totals. No install id, install-day hash, batch id, version, platform, provider, errors, prompts, keys or paths. A source-scan test keeps the profile modules away from the anonymous identity.
  • Deletion. Disconnect is always allowed (agents, stopped proxy, telemetry off) and deletes the profile and every row on the collector; if the collector is unreachable it stays pending and the proxy retries.
  • Collector (services/telemetry-collector): POST /v1/profiles, POST /v1/profiles/me/days, GET|DELETE /v1/profiles/me, GET /v1/leaderboard, GET /v1/profiles/by-handle/<h>. Profile tables never join the anonymous ones. Abuse controls: domain-separated rate-limit hashes, reserved handles, consistency and daily caps, first-write-wins per day, no backfill, RANK() ranking, a hidden moderation flag, squatting prune, noindex, and no CORS preflight for bearer routes. Nothing is deployed.
  • opencodex.me/dashboard gains a leaderboard (day/week/month, tokens/requests) and a ?profile=<handle> personal-rank view in 8 locales, rendered with textContent only.
  • Docs: telemetry guide section, privacy line in README + 7 translations (manifest hashes updated), code-signing pages en/tr, AGENTS_INSTALL.md consent rule, SPONSORS.md, structure/telemetry.md, collector README. No installer change, since the profile is opt-in.

Design and audit record: devlog/_plan/261109_telemetry_personal_rank/.

Screenshots (dashboard Home, isolated proxy; the connected view shows "Rank is unavailable" because no collector is deployed yet):

Connect form
Connected profile

Verification

All commands ran in isolated homes (env -i, temp HOME/OPENCODEX_HOME/CODEX_HOME/CLAUDE_CONFIG_DIR).

  • New tests: tests/usage/usage-telemetry-profile-{contract,sync,collector}.test.ts, tests/server/server-telemetry-profile-routes.test.ts, tests/cli/cli-telemetry-profile.test.ts, tests/ci-workflows/docs-telemetry-leaderboard.test.ts and gui/tests/telemetry-profile-card.test.tsx. All pass. Each review fix has a test that failed before it. Collector mutation checks (ROW_NUMBER for RANK, allowing backfill, dropping the hidden filter) were each caught.
  • bun run typecheck, bun run lint:gui, bun run build:gui, docs-site astro build (internal link and fragment check), bun run privacy:scan, bun run structure:check, bun run skill:surface:check: pass. GUI suite: 4109 pass. Test-layout, file-size ratchet and core/Lab boundary tests pass, and no file-size cap was raised.
  • Full bun run test (before the review fixes): passes except four tests, all unrelated to this branch:
    • three codex-cli-update-apply cases, which fail because this machine has live Codex sessions (blocked_active_session);
    • release-version-line, which fails because the stack base's package.json (2.99.0) equals an already-released tag. This PR does not touch package.json.
  • Reviews (devlog/_plan/261109_telemetry_personal_rank/, decisions recorded per finding):
    • Plan audit: NEAR-PASS, folded.
    • Code review round 1 (hyds/deepseek-v4.1-flash-free): REQUEST-CHANGES, 1 P1 + 2 P2, all fixed with failing-first tests (b42784e33c, 11f62c7acc).
    • Separate privacy/security review (mistral/deepseek-v4-1-flash-1017): PASS-WITH-NITS; the P2 (disconnect reported done while the secret file remained) is fixed in 31ed41f70b; 4 of 5 P3s folded.
    • Round 2: APPROVE-WITH-NITS; P2 (leaderboard cache across UTC midnight) fixed in 8d42472900, one P2 rebutted with a passing test.
    • Round 3: APPROVE-WITH-NITS; P2 (a hide took up to 5 minutes to apply) fixed in ca51591a38.
    • Final fresh review on head ca51591a38: APPROVE-WITH-NITS, no P0/P1, its single P2 rebutted (a no-usage day has no family rows to lose, and only completed UTC days are read); 3 P3s deferred.
    • The command-code DeepSeek route answered HTTP 401 on every attempt, so reviews ran on the hyds and mistral routes.
  • CI: react-doctor flagged a response read before its status check, fixed in 1a1d36f8d7. test 5/5 fails only release-version-line, which is inherited from the stack base: its package.json is 2.99.0, already a released tag, while dev is at 2.100.0. It clears once the stack takes dev.
  • After the stack landed: merged dev as 632f59b0a6, taking dev for every file and re-applying only this PR's changes. The diff against dev matches the original layer exactly (78 files, +5695/−27). Re-verified in isolated homes: typecheck, 291 focused and guard tests, 37 GUI telemetry tests, GUI lint and build, docs-site build, privacy:scan, structure:check and skill:surface:check all pass. tests/usage/usage-telemetry-profile-lifecycle.test.ts (added in 5ede3c63bf) proves that installs without a profile start no profile loop, with telemetry on or off; removing the guard makes it fail.
  • NOT RUN: deploying the collector; real D1 (tests use bun:sqlite through the existing D1 harness); macOS/Windows CI (left to hosted CI). The branch was pushed with --no-verify.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Roadmap unit for a six-layer stack: client telemetry core, GUI consent popup, Cloudflare Worker collector, telemetry guide, and the public /dashboard page. Docs only.
# Conflicts:
#	README.md
#	docs-site/src/content/docs/reference/code-signing.md
#	docs-site/src/content/docs/tr/reference/code-signing.md
#	readme/README.fr.md
#	readme/README.ja.md
#	readme/README.ko.md
#	readme/README.ru.md
#	readme/README.tr.md
#	readme/README.zh-CN.md
#	readme/README.zh-TW.md
#	readme/i18n-manifest.json
#	structure/telemetry.md
…ut docs-site deps

The root docs-telemetry-dashboard test imports client.ts; the root CI job does not install docs-site dependencies, so the static echarts import failed to resolve. ECharts now loads on first chart render as its own chunk.
…ever sent

A proxy started under OCX_TELEMETRY=0 reads the env once and runs no ticks, so it
records ineligibleThroughDay only for its start day. These tests drive the real
background lifecycle with a mocked clock for an opt-out period of several days,
ended by a crash and by a clean stop. They fail on this commit: the first normal
tick sends 2026-11-03, a day the proxy ran opted out.
…il it ends

Observing OCX_TELEMETRY=0 or DO_NOT_TRACK=1 now also persists an
envOptOutActiveSince marker in telemetry-state.json. While it is set no day is
eligible. The opted-out proxy's clean stop, or the first tick that runs without the
opt-out, closes the period: every day through the closing day stays ineligible and
the marker is cleared. A crashed opted-out proxy therefore costs at most the
restart day and never leaks its opt-out days.

SPONSORS.md no longer says opencodex collects no telemetry.
…ed on other platforms

From the final fresh review of #8348 (P1): APPLIED_STAMP_VALUE, APPLIED_USER_STAMP_VALUE, Scope::User and Scope::applied_value are used only by the Windows registry reader and the unit tests, so cargo clippy --all-targets -D warnings (the Linux desktop shell CI job) failed with four dead-code errors on non-Windows targets. They carry the same cfg_attr allow as the other Windows-only names. Real-crate host clippy: 4 errors before, clean after.
…vacy statements, install rules and structure doc
…used days aside, and let disconnect clear an unreadable profile file
…le remains; clarify status and indexing wording
@lidge-jun lidge-jun added the ci:full Run the four full Ubuntu test shards on this pull request label Nov 8, 2026
@coderabbitai

coderabbitai Bot commented Nov 8, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions

github-actions Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

✅ Deterministic PR hygiene checks passed.

@github-actions github-actions Bot added the enhancement New feature or request label Nov 8, 2026
…ending deletion from the reply, and tighten preview and loop start
…d read; refuse connect before prompting when telemetry is off
…board

# Conflicts:
#	devlog/_plan/261107_telemetry_dashboard/060_review_round.md
#	docs-site/src/content/docs/guides/telemetry.md
#	readme/i18n-manifest.json
#	scripts/test-layout/layout-usage.json
#	src/telemetry/consent.ts
#	src/telemetry/routes.ts
#	src/telemetry/sender.ts
#	src/telemetry/state.ts
#	structure/manifest.json
#	structure/telemetry.md
#	tests/config/telemetry-live-save.test.ts
#	tests/fixtures/test-layout-expected.json
#	tests/server/server-telemetry-routes.test.ts
#	tests/usage/usage-telemetry-client.test.ts
…o goat/telemetry-07-personal-rank

# Conflicts:
#	readme/i18n-manifest.json
Base automatically changed from goat/telemetry-06-dashboard to dev November 9, 2026 05:40
…onal-rank

The stack landed as squash commits, so this merge takes dev for every file and re-applies only this
PR's own changes (879a507..e91c0db) on top; no older stack copy is kept.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:full Run the four full Ubuntu test shards on this pull request enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Opt-in public profile and personal usage rank on top of anonymous telemetry

1 participant