release: promote investigations and billing to production

.agents/skills/databuddy-internal/SKILL.md

@@ -19,6 +19,7 @@ Keep additions **minimal**: one bullet, a new `rg` hint, or a routing note—eno
 
 ## Quick Map
 
+- Before any PR merge, follow the AGENTS.md review-feedback gate: wait for configured reviewers on the final head, read all comment/review/thread pages, address each finding with evidence, and re-fetch to verify no unresolved feedback. Review bots can finish several minutes after a draft becomes ready; green CI does not establish completed review.
 - Prod infrastructure repo is local at `/Users/iza/Documents/GitHub/databuddy-infra` (`databuddy-analytics/infra`); ClickHouse cluster inventory is `clickhouse/ansible/inventory.yml`, not `/Users/iza/Dev/Databuddy/infra` or `DatabuddyOPS`.
 - Never use production/customer data as tests, fixtures, snapshots, examples, or copied output. Tests must use placeholders/mocks only (example.com, example IDs). If production ClickHouse is queried for investigation, summarize anonymized aggregates and do not paste customer domains, client IDs, emails, or other identifiers into code or responses.
 - `@databuddy/test/env` targets local `databuddy_test` unless `CI=true`, so a normal `db:push` may update a different database; sync that test database explicitly before debugging removed-column failures.
@@ -29,6 +30,7 @@ Keep additions **minimal**: one bullet, a new `rg` hint, or a routing note—eno
 - Local E2E dashboard smokes that need `/api/test/e2e/*` should start the API/dashboard directly (or through Playwright's webServer command), not via `bun run dev:dashboard`; Turbo runs in strict env mode and drops `DATABUDDY_E2E_MODE`/`DATABUDDY_E2E_TEST_KEY` unless they are added to `turbo.json` `globalEnv`.
 - Dashboard Playwright public/demo analytics specs call API `/v1/query` anonymously from the browser; keep `DATABUDDY_E2E_MODE` query behavior isolated from production rate limits so CI retries do not exhaust `anon:unknown`.
 - `apps/api`: Elysia API on port `3001`
+- API tests use Vitest through `bun run test` inside `apps/api`; use Vitest test imports rather than `bun:test` in that package.
 - Public REST docs live in `apps/api/src/rpc/openapi.ts`: `/spec.json` is the generated spec, `/` is the reference UI, and hiding a router there also makes its top-level REST paths return 404 because `/*` uses the same filtered docs router.
 - `apps/slack`: Slack agent adapter; Slack installs resolve through org-scoped DB integration records, not a single env bot token/default website. Agent calls use the org-scoped internal principal synthesized from the active integration in `slack/installations.ts`, never a global internal secret.
 - Slack OAuth lives in `apps/api`, but slash commands/events require `apps/slack` to be running too; local `bun run dev:dashboard` runs dashboard + API only, so use `bun run dev:slack` when working on Slack. The Slack package scripts read the root `.env`.
@@ -44,7 +46,7 @@ Keep additions **minimal**: one bullet, a new `rg` hint, or a routing note—eno
 - `SPEC.md` is the intelligence product contract. `insight_observations` is the readable Insights history; `analytics_insights` is the durable investigation projection. The agent outcome owns brief publication and `act`/`ask` promotion; do not replace either with frontend heuristics or collapse the feed into cases. Do not add a parallel agent, evidence API, fixed query choreography, or action-specific lifecycle.
 - Insights quality reviews must compare fresh baseline/candidate outputs and lead with the product verdict and concrete examples. Score usefulness, noise, reading effort, and retained useful findings separately from code tests and contract passes; preserve interrupted attempts instead of reporting retries as an uninterrupted pass rate.
 - Insights RPC helpers that take `{ context, ...input }` must strip `context` before parsing a `.strict()` Zod input schema (same pattern as `appendInvestigationReply` / `applyInsightGoalAction`); otherwise CI fails with `Unrecognized key: "context"`.
-- `insights.history` / MCP `list_investigations` hide cases while a reply is `queued`/`running` (action-inbox verification); tests must list before reply or expect an empty list while verifying.
+- `insights.history` / MCP `list_investigations` hide cases while analysis or verification is queued/running; included clarifications use saved evidence and must not hide or mutate the case.
 - When reporting what an organization can see in Insights, follow the `insights.brief`/`history` visibility rules instead of counting `analytics_insights`; the projection can contain legacy rows without a readable or published `insight_observations` turn.
 - Production insight shadows must freeze `--reference-time`, retain a tool-name trace, and pass available GitHub context before supporting quality claims. Postgres and ClickHouse are read-only, but connector token refreshes or cache writes can still occur; never describe the whole run as zero-write.
 - Automatic investigations have one organization-wide schedule (`off`, `daily`, or `weekly`) and one organization-wide delivery set; website selection is only for manual runs. Do not reintroduce per-website overrides, hourly/custom cadence, or cron input.
@@ -142,6 +144,7 @@ Read [codebase-map.md](./references/codebase-map.md) when you need deeper routin
 - Funnel rows keep the action menu outside the main toggle button; put row padding on the sibling `Button`, not only on `List.Row`, so the visible row surface is clickable without nesting buttons.
 - Demo website navigation must be public-safe and route-backed; hide sensitive, configuration-heavy, or unavailable website features such as Agent, Feature Flags, Revenue, Users, Realtime, and website Settings instead of inheriting the full website nav. Goals and Funnels may be public demo surfaces, but keep them read-only.
 - Dashboard definitions for feature flags and target groups are admin surfaces; do not expose even sanitized rows to demo-tier/public website access.
+- Billing settings must render details only while their switch is on; hidden grid rows can retain blank height and focusable controls. Autumn `overageLimit` is additional feature units, not USD. Use the native `databunny_chat` flag to hide legacy credit purchases and guards for included-chat accounts; an investigation balance alone does not establish chat terms.
 - Insights history is grouped by its backend-owned subject key in the RPC layer so every client sees one current row per investigation; reads must not invoke AI generation.
 - An executable Insight action must apply its stored mutation and enqueue verification transactionally; never ask a teammate to mark it done manually. Show a verified result only after measured evidence supports it.
 - Insights case pages should not explain their own structure (for example, "context, investigation, and verification in one thread"). Keep only evidence, current state, and actionable controls visible.
@@ -166,7 +169,8 @@ Read [codebase-map.md](./references/codebase-map.md) when you need deeper routin
 - Start in `apps/api/src`
 - Shared API contracts and procedure logic live in `packages/rpc`
 - Prefer changing shared router logic in `packages/rpc` rather than duplicating validation in the dashboard
-- Investigations run in `apps/insights`; RPC only reads cases and accepts durable replies. Case identity is `websiteId|subjectKey`, where the backend owns the subject key. Persist a new observation for each turn while updating the existing insight row. The stored `changePercent` is already signed.
+- Saved investigation tool evidence must use typed, positive field allowlists; do not persist arbitrary tool outputs or rely on generic secret-pattern redaction. Preserve exact measurement scope, and record omissions instead of reconstructing missing raw evidence.
+- Investigations run in `apps/insights`; RPC only reads cases and accepts durable replies. Case identity is `websiteId|subjectKey`, where the backend owns the subject key. New analysis appends an observation; a clarification stores its answer on the reply and reads the originating observation's saved evidence without changing case state. The stored `changePercent` is already signed.
 
 ### Ingestion and analytics pipeline
 
@@ -177,9 +181,16 @@ Read [codebase-map.md](./references/codebase-map.md) when you need deeper routin
 
 ## Billing (Autumn)
 
+- Business includes 100 investigations/month and Scale 500, then $1 per extra. Lead pricing cards with these allowances; keep investigation definitions in one short FAQ. A pushed config is not an active offer: verify the serving deployment and native Autumn plan version before claiming the change is live.
+
+- Billing-control read/modify/write must use a strict native client and verify the customer ID before merging settings. SDK `getOrCreate` can fail open with empty controls, causing a later successful update to erase unrelated saved limits.
+
+- Autumn owns all investigation allowances, charges, reservations, prices, and invoicing. Do not store investigation billing state in Postgres tables, fields, or a billing outbox. Use existing product identities and completion state for provider retry references. Verify deployed entitlement support before syncing live plans.
+
+- Autumn catalog updates must send complete mutable plan fields, including explicit `addOn`, `autoEnable`, price, and description: omitted provider fields can reset flags, erase descriptions, or create a free new version. Preserve live legacy economics and verify exact before/after provider readback; passing SDK/CLI validation does not establish provider defaults. Do not overwrite unrelated live credit-schema drift during a pricing sync.
 - Retried insight jobs must persist immutable external delivery effects (currently Slack) before calling providers and reuse the effect ID as the provider idempotency key. An insight observation is product memory, not a delivery checkpoint.
-- Intelligence pricing should use the existing token-cost-backed `agent_credits` and top-up flow; do not invent per-site or "monitored product" billing without explicit product selection and runtime enforcement.
-- Transactional billing email identity has three separate concepts: Autumn customer/billing owner, organization, and actual `to` recipient. Only personalize from the actual recipient record; if it is unavailable, omit the greeting rather than using the owner name. Keep `agent_credits` as an internal feature ID, but describe it to customers as investigation credits and explain that deeper investigations, replies, and rechecks can use more credits.
+- Investigation plans use monthly counts from `INVESTIGATION_ALLOWANCES` and $1 per additional completed result through Autumn `investigation_runs`; never market token credits or an access gate as an investigation allowance. Render the actual attached allowance/reset separately from prepaid balances, and verify catalog, customer subscription, pricing cards, billing, and machine-readable docs together. Clarifications and repair verification are included. Validate explicit analysis consent at the API boundary and reserve one unit in Autumn before analysis; settle only after a readable complete result is persisted, reusing product identity on retries. Existing customers without the entitlement retain legacy `agent_credits` terms; do not convert their balances implicitly.
+- Transactional billing email identity has three separate concepts: Autumn customer/billing owner, organization, and actual `to` recipient. Only personalize from the actual recipient record; if it is unavailable, omit the greeting rather than using the owner name. Distinguish fixed-price investigations from legacy credits in billing copy.
 - `autumn-js` v1.2.2+ — import `autumnHandler` from `autumn-js/fetch` (NOT `autumn-js/elysia`, that export was removed in v1.0)
 - For Elysia, mount with `.mount(autumnHandler(...))` — NOT `.use()`
 - `identify` callback receives `(request: Request)` directly, not `({ request })`
@@ -193,14 +204,15 @@ Read [codebase-map.md](./references/codebase-map.md) when you need deeper routin
 
 ### Database work
 
-- Postgres schema: `packages/db/src/drizzle/schema.ts`
+- Postgres schemas: `packages/db/src/drizzle/schema/` (`index.ts` barrel)
 - Relations: `packages/db/src/drizzle/relations.ts`
 - Drizzle client: `packages/db/src/client.ts`
 - Production `DATABASE_URL` may already target PgBouncer; inspect both the process pool and PgBouncer queues before attributing API timeouts to PostgreSQL.
 - `pg.Pool` already grows lazily from zero to its configured `max`; do not replace it with one `Client` to address acquisition timeouts, because that serializes queries. Keep a bounded pool, tune its acquisition timeout, and monitor `waitingCount`.
 - ClickHouse helpers and schema: `packages/db/src/clickhouse/*`
 - `ch:check` is package-scoped; run `cd packages/db && bun run ch:check`, not the root script runner.
 - After schema changes, use the repo db scripts rather than ad hoc commands
+- PostgreSQL deploys use `packages/db db:push` through `init.Dockerfile`; register new schema files in `packages/db/drizzle.config.ts`. `packages/migrate` transforms SDK source and is not a database migration runner.
 - A shipped ClickHouse table change needs a tracked forward migration alongside
   its reference DDL: bootstrap `CREATE ... IF NOT EXISTS` does not migrate
   deployed tables, and Keeper-path or sort-key changes need a shadow-table

.agents/skills/databuddy-internal/references/codebase-map.md

@@ -9,6 +9,8 @@ Use this file when the task spans multiple packages or when the right edit locat
 - Theme tokens and WCAG-oriented contrast: `apps/dashboard/app/globals.css` (`:root`, `.dark`)
 - Next.js dashboard application
 - Default dev port: `3000`
+- For local app review, use `bun run dev:dashboard` with the same existing root `.env` across services; do not blank billing or AI credentials and present the resulting environment as a working app.
+- Include interaction feedback in billing/pricing work: short transform/opacity reveals, native pending buttons with accessible labels, and reduced-motion support. Keep disabled billing details unmounted and verify motion in the local browser.
 - Talks to the backend through oRPC using [`apps/dashboard/lib/orpc.ts`](/Users/iza/Dev/Databuddy/apps/dashboard/lib/orpc.ts)
 - **Status pages:** Public page lives under `app/status/[slug]`; **dashboard** edit surface is `app/(main)/monitors/status-pages/[id]/page.tsx`. For stable chrome (no tab strip pop-in), keep `Tabs` always mounted and gate **content** with `useFeatureAccess` + `FeatureLockedPanel`—do not wrap the whole `Tabs` in `FeatureAccessGate` with a list-only `loadingFallback`. Reserve `PageHeader` `right` with skeletons until `statusPage` loads; use a static `description` and breadcrumb fallback `"Status page"` so the header does not reflow on fetch.
 - Typical work:
@@ -54,6 +56,7 @@ Use this file when the task spans multiple packages or when the right edit locat
 - Next.js docs site using Fumadocs
 - Default dev port: `3005`
 - Good place for product docs, guides, and marketing-adjacent content with app integrations
+- `SciFiButton` styles its inner control separately from the corner-frame wrapper. In stretched pricing-card layouts, use `className="w-full"` and verify their bounds match at desktop and mobile widths.
 - Pricing for agents: `public/pricing.md`; `GET /api/pricing` (JSON or markdown via `Accept`); middleware rewrites `/pricing` → `/api/pricing` when markdown wins.
 - Cookie cost calculator: [`apps/docs/app/(home)/calculator/`](/Users/iza/Dev/Databuddy/apps/docs/app/(home)/calculator/) — uses **visitor data loss rate** (consent / analytics visibility), not “banner bounce”; literature band 40–70% in `calculator-engine.ts`.
 
@@ -73,7 +76,7 @@ Use this file when the task spans multiple packages or when the right edit locat
   - Postgres schema and relations
   - ClickHouse client and schema
 - Key files:
-  - [`packages/db/src/drizzle/schema.ts`](/Users/iza/Dev/Databuddy/packages/db/src/drizzle/schema.ts)
+  - [`packages/db/src/drizzle/schema/index.ts`](/Users/iza/Dev/Databuddy/packages/db/src/drizzle/schema/index.ts)
   - [`packages/db/src/drizzle/relations.ts`](/Users/iza/Dev/Databuddy/packages/db/src/drizzle/relations.ts)
   - [`packages/db/src/client.ts`](/Users/iza/Dev/Databuddy/packages/db/src/client.ts) — strips `sslrootcert=system` from `DATABASE_URL` before `pg` Pool: libpq uses it for the OS trust store, but node-postgres treats `sslrootcert` as a file path and throws `ENOENT` on path `"system"`.
   - [`packages/db/src/clickhouse/client.ts`](/Users/iza/Dev/Databuddy/packages/db/src/clickhouse/client.ts)

.github/workflows/ci.yml

@@ -81,6 +81,7 @@ jobs:
           restore-keys: ${{ runner.os }}-bun-
       - run: bun install --frozen-lockfile --ignore-scripts
       - run: bun run --cwd apps/docs postinstall
+      - run: bun run --cwd apps/docs lint:links
       - run: bun run check-types
       - name: ClickHouse types match DDL
         run: bun run --cwd packages/db ch:check

AGENTS.md

@@ -160,6 +160,7 @@ For picker controls, use the component that matches the interaction:
 - **Start fresh**: Check for an existing PR that owns the same surface, public contract, schema, or deployment configuration, then create the branch from an up-to-date `origin/staging`. Do not use an unmerged feature branch as a base unless the dependency is explicit, approved, and named as `Depends on #…` in both PRs.
 - **Make ownership visible**: Push and open a draft PR against `staging` once the slice has a first commit. State its scope, dependencies, and known overlaps.
 - **Keep integration linear**: Rebase a slice onto current `origin/staging` before it is ready for review; do not merge `staging` into the slice merely to refresh it. Request fresh review when a rebase changes reviewed code.
+- **Resolve all review feedback before merging**: Mark the PR ready and wait for configured reviewers to finish on the final head; green CI alone is insufficient. Read every page of general comments, reviews, and inline threads, including outdated threads. Fix actionable findings or document a supported reason for declining them, then resolve each thread. Immediately before merging, re-fetch feedback and verify zero unresolved threads and no unaddressed comments or pending reviews. Never merge immediately after marking a draft ready or pushing review fixes while reviewers are still running.
 - **Isolate parallel work**: Use one worktree per active branch. Never let two agents or contributors mutate the same branch or reuse a task branch for a different concern.
 - **Retire completed work**: Merged PR source branches are automatically deleted. Delete closed PR branches manually, remove clean finished worktrees, and create a new branch from current `staging` for any follow-up—never revive or repurpose an old PR branch.