Autorecovery reads a different ClickHouse database than the worker

[MicroMilo]

Summary

The worker's main ClickHouse client, environment example, trace-context reader, and backfill reader consistently use CLICKHOUSE_DB with a superlog fallback. Autorecovery creates a separate production client that first accepts an otherwise nonstandard CLICKHOUSE_DATABASE variable and falls back to olly. Thus two runtime roles in the same worker select different databases under the repository's unset-variable defaults.

Impact: When neither database variable is set, normal worker ingestion and queries use superlog while autorecovery queries olly. In the repository's local stack, this makes autorecovery query a database that is not the configured worker store and typically does not exist; candidate reads fail and the recovery loop produces no proposals. If both databases exist, autorecovery reads the wrong data set. Setting only the autorecovery-supported CLICKHOUSE_DATABASE alias also splits the roles.

Code path

Checked against current main at commit 5db3e5842bff9f3b4df005542d80da4608cf6817.

• apps/worker/src/index.ts:34-45
• apps/worker/src/autorecovery/metrics-repository.ts:187-205
• apps/worker/.env.example:1-5

Steps to reproduce

Validation level: current HEAD supplemental dynamic witness reproduced.

node --input-type=module -e 'import fs from "node:fs"; const read=(p)=>fs.readFileSync(p,"utf8"); const fallback=(p,key)=>{const m=read(p).match(new RegExp(`process\\.env\\.${key}\\s*\\?\\?\\s*"([^"]+)"`)); return m?.[1]??"<none>"}; const auto=read("apps/worker/src/autorecovery/metrics-repository.ts").match(/process\.env\.CLICKHOUSE_DATABASE\s*\?\?\s*process\.env\.CLICKHOUSE_DB\s*\?\?\s*"([^"]+)"/)?.[1]; console.log(JSON.stringify({workerMainDefault:fallback("apps/worker/src/index.ts","CLICKHOUSE_DB"),traceContextDefault:fallback("apps/worker/src/infra/clickhouse/trace-context.ts","CLICKHOUSE_DB"),backfillDefault:fallback("apps/worker/scripts/backfill-otel-exceptions.ts","CLICKHOUSE_DB"),autorecoveryDefault:auto},null,2))'

Observed output:

{
  "workerMainDefault": "superlog",
  "traceContextDefault": "superlog",
  "backfillDefault": "superlog",
  "autorecoveryDefault": "olly"
}

Neighboring control:

node --input-type=module -e 'import fs from "node:fs"; const read=(p)=>fs.readFileSync(p,"utf8"); const val=(p,key)=>read(p).match(new RegExp(`process\\.env\\.${key}\\s*\\?\\?\\s*"([^"]*)"`))?.[1]??"<none>"; for (const key of ["CLICKHOUSE_URL","CLICKHOUSE_USER","CLICKHOUSE_PASSWORD"]) console.log(`${key}: main=${JSON.stringify(val("apps/worker/src/index.ts",key))} autorecovery=${JSON.stringify(val("apps/worker/src/autorecovery/metrics-repository.ts",key))}`)'

Control output:

CLICKHOUSE_URL: main="http://localhost:8123" autorecovery="http://localhost:8123"
CLICKHOUSE_USER: main="default" autorecovery="default"
CLICKHOUSE_PASSWORD: main="" autorecovery=""

Expected behavior

The documented/source-grounded contract should hold without the drift described above.

Actual behavior

Autorecovery defaults to a different ClickHouse database from every neighboring worker consumer.

Existing coverage

I checked the current issue and PR lists for overlapping titles/root-cause keywords before filing this. I did not find an item covering this same root cause.

Suggested fix

Centralize worker ClickHouse configuration so autorecovery and the primary worker resolve the same database variable and fallback.

Suggested tests

• Add a regression test for the reproduction above.
• Add a neighboring control assertion so the intended non-bug path remains covered.

---

Submitted with Codex.

[Keesan12]

Recovery is only useful when it is recovering the same reality the worker was operating in. Otherwise it is just a second system confidently fixing the wrong thing.

We're building MartinLoop around that hard edge for coding agents: a run needs a durable record of state, budget, outcome, and failure reason before anyone or anything resumes it. Blind recovery is how quiet failures get expensive.

Would love to know what invariant you would put ahead of speed here. We are building in public at https://github.com/Keesan12/martin-loop