feat(#126): migrate Langfuse to JS SDK v5 for the v4 observations model

[alexosugo]

Description

Stacked on #127: migrates the Langfuse tracing added there from the legacy langfuse v3 npm package to the OpenTelemetry-based JS SDK v5 (@langfuse/tracing, @langfuse/otel, @langfuse/client), targeting Langfuse's v4 observations-first data model. The legacy batch-ingestion API that v3 uses is removed from Langfuse Cloud on 2026-11-16.

• src/observability now exposes withTrace, observeGeneration, scoreTrace, and shutdownLangfuse. Session id, tags, and metadata are propagated to every observation, so session and tag aggregations include generation cost.
• The root observation carries the overall input/output; the deprecated trace-level input/output setters are not used.
• Trace ids are generated per run instead of being derived from the PR number, so reprocessing a PR produces a distinct trace.
• Generations record usageDetails on the API path and costDetails on the Claude CLI path.
• Span export uses a 3s timeout; score requests use the @langfuse/client default of 60s (5.11.1 ignores its timeout option). Langfuse is shut down once per run, and flush failures only log a warning.
• Tracing is off unless LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY are both set.
• docs/observability.md is updated for the new API.

Verification:

• tsc, lint, and unit tests (1133 passing) run locally.
• One live pipeline run on the API path (OpenRouter), checked in Langfuse Cloud: the trace, session, tags, distill-outcome score, token usage, and inferred cost all arrived.
• The Claude CLI path was not verified live here: an existing bug in the parser for the CLI's JSON output blocked it. feat(#126): trace research/dev runs and record code-gen model, usage and cost #163 fixes that parser.

Part of the #127 → #162 → #163 stack; merge after #127.

#126

Code review checklist

• Readable: Concise, well named, follows the style guide, documented if necessary.
• Tested: Unit and/or e2e where appropriate
• Backwards compatible: Works with existing data and configuration or includes a migration. Any breaking changes documented in the release notes.

License

The software is provided under AGPL-3.0. Contributions to this project are accepted under the same license.

[sugat009]

Thank you for the migration. The v5 wiring is correct. I tested it against a local fake Langfuse.

Two items block the merge: index.ts:35 and index.ts:24. Both have one-click suggestions.

CI is green. With all the suggestions applied, 1023 tests pass locally.

[sugat009]

Thank you. All 7 round-1 points are fixed. I checked each one on 012d77e:

• A failed span export now logs a warning, the score still goes out, and the run keeps its exit code (0, or 2 on a rate limit).
• With no keys, or with one key only, nothing is sent.

Local gates pass: fresh npm ci, build, lint, validate-schema, and test:coverage with 1023 tests. The CI test job did not run on this head, because it runs only for PRs to main.

note (non-blocking): .env.example:66-67 sets placeholder keys (pk-lf-...). The README tells new developers to copy that file and edit only ANTHROPIC_API_KEY. The new key check then sees two keys, and tracing stays on with invalid keys. Commenting out those two lines would close this. The file is outside this diff, so I note it here.