Fix model name mismatch between sample.env and Bicep + add validation

[diberry]

Problem

\sample.env\ referenced outdated models (\gpt-4o, \ ext-embedding-ada-002) and API versions (\2024-10-21), while \infra/main.bicep\ deploys \gpt-4.1-mini\ and \ ext-embedding-3-small\ with API version \2025-01-01-preview. The source code fallback defaults in \llm.ts\ also used the stale values. This three-way inconsistency would confuse developers doing local setup.

Additionally, \sample.env\ had no \SET_PASSWORDLESS\ variable even though Bicep deploys with managed identity auth.

Changes

Config fixes

• sample.env: Updated model names, API versions, and added \SET_PASSWORDLESS=true\
• llm.ts: Updated fallback defaults from \gpt-4o/\2024-10-21\ → \gpt-4.1-mini/\2025-01-01-preview\

Drift prevention (new)

• **\scripts/validate-config.mjs**: Node.js script that parses \sample.env, \infra/main.bicep, and source code fallbacks, then compares model names and API versions. Exits non-zero on mismatch.
• **.github/workflows/validate-config.yml**: GitHub Action that runs the validation on PRs touching any of these files.
• **
  pm run validate:config**: npm script for local use.

How it prevents future drift

If someone changes the model in Bicep but forgets to update \sample.env\ (or vice versa), CI will fail with a clear error showing which values are out of sync.

[diberry]

❌ Reviewed by Piggy (Quality Reviewer) — REJECTED (needs 1 fix)

Summary

Good intent — fixes real config drift and adds CI guard. But the PR introduces a new inconsistency it should have caught, and the validation script has coverage gaps.

🔴 Blocking Issue (1)

API version fallback missed in packages/langgraph-agent/src/config/llm.ts

• Line 4 still has 2024-10-21 as fallback
• packages-v1/langgraph-agent/src/azure/llm.ts is correctly updated to 2025-01-01-preview
• sample.env and infra/main.bicep both say 2025-01-01-preview
• This is exactly the three-way drift the PR claims to fix

Fix: Change packages/langgraph-agent/src/config/llm.ts line 4 from 2024-10-21 to 2025-01-01-preview

🟡 Non-Blocking Improvements (2)

1. Validation script only checks packages-v1/ source files

The CONFIG_CHECKS array points sourceFile only to packages-v1/langgraph-agent/src/azure/llm.ts. It does not validate:

• packages/langgraph-agent/src/config/llm.ts (has model + API version fallbacks)
• packages/langgraph-agent/src/config/embeddings.ts (has model fallback)

The blocking bug above proves this gap — the script would pass while drift exists.

2. CI workflow path triggers missing packages/ paths

.github/workflows/validate-config.yml watches packages-v1/**/azure/llm.ts and packages-v1/**/azure/embeddings.ts but not packages/**/config/*.ts. Changes to packages/ source won't trigger validation.

✅ What's Correct

• Model name updates (gpt-4o to gpt-4.1-mini, text-embedding-ada-002 to text-embedding-3-small) match Bicep
• maxTokens fix (100 to 1000) applied to both packages/ and packages-v1/
• SET_PASSWORDLESS=true addition to sample.env is appropriate
• actions/checkout@v6 confirmed valid (v6.0.2 is latest)
• Validation script parsers are well-written
• package.json npm script addition is clean

Pre-existing (not blocking)

• Bicep output AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME maps to embeddingApiVersion instead of embeddingModelName (infra/main.bicep line 344). Not introduced by this PR.

[diberry]

Piggy (Quality Reviewer) — ✅ Approved on re-review

All 3 previously identified issues verified fixed:

1. API version updated to 2025-01-01-preview ✅
2. Validation script now covers packages/ ✅
3. CI triggers include config paths ✅

[diberry]

Kermit (Lead) — ✅ Approved

Verdict: This PR directly solves a real developer trap — the three-way config mismatch between sample.env, Bicep, and source fallbacks would cause runtime errors for anyone following the setup guide.

Review by area:

1. Reader impact: Strong positive. Developers copying sample.env would have gotten gpt-4o while Bicep deploys gpt-4.1-mini → model-not-found errors at runtime. Fixed across all three layers now.

2. Validation script: Excellent pattern. Clean parsers, clear output, proper exit codes. CI integration via path-filtered workflow is smart — only runs when config files change. Minor gap: README doesn't mention npm run validate:config. Consider adding a one-liner in the Contributing or Development section so first-time contributors discover it.

3. sample.env clarity: Improved. SET_PASSWORDLESS=true with the auth-mode comment helps first-time users understand the managed identity default. Model names now match what Bicep actually deploys.

4. Bicep/infra alignment: All four config values now match:

   • GPT: gpt-4.1-mini + 2025-01-01-preview ✅
   • Embedding: text-embedding-3-small + 2023-05-15 ✅
   • maxTokens: 100 → 1000 (fixes likely truncation bug) ✅

5. Scope: Well-scoped. Three related concerns (fix mismatch, prevent drift, add missing env var) all tied to config consistency.

Non-blocking observations for follow-up:

• Pre-existing Bicep bug (out of scope): infra/main.bicep:344 sets AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME = embeddingApiVersion — deployment name is accidentally assigned the API version variable. Worth a separate fix.
• README gap: Document the validation script so contributors know it exists.
• Reusable pattern: This validate-config approach is worth adopting in other Azure-Samples repos. Consider linking it from the repo's contributing guide.

[diberry]

Gonzo (Tech Researcher) — ✅ Approved

Review scope: Model names, API versions, Bicep correctness, SDK compatibility, validation script logic.

1. Model name — gpt-4.1-mini ✅
Confirmed GA on Azure (version 2025-04-14), available in 28 regions via GlobalStandard SKU. Correctly replaces stale gpt-4o default. Bicep model version '2025-04-14' is accurate.

2. API version — 2025-01-01-preview ✅ (with note)
Valid Azure OpenAI data-plane preview version (confirmed in azure-rest-api-specs). Note: not the latest preview (2025-04-01-preview exists), but functional and consistent. Not blocking.

3. Bicep resources ✅

• gpt-4.1-mini deployment: correct model name, version, GlobalStandard SKU, capacity 50
• text-embedding-3-small: correct model, version '1', GlobalStandard SKU
• AVM module versions current; S0 Cognitive Services SKU correct
• Container App env vars properly plumbed from Bicep params

4. SDK compatibility ✅
@langchain/openai AzureChatOpenAI works with any Azure-deployed model. The pattern \�zureOpenAIApiDeploymentName: model\ (deployment name = model name) is standard Azure convention and works correctly with gpt-4.1-mini.

5. Embedding model update ✅
text-embedding-ada-002 → text-embedding-3-small is correct. The hardcoded \dimensions: 1536\ in packages-v1/embeddings.ts matches text-embedding-3-small's default output dimensionality.

6. maxTokens fix ✅
100 → 1000 in both packages-v1 and packages llm.ts. Matches sample.env and Bicep. Old value of 100 was a bug — too low for RAG answer generation.

7. Validation script ✅

• parseSampleEnv: correct env parsing with quote stripping
• parseBicepParams regex: correctly matches \param string = ''\ patterns
• parseSourceFallbacks regex: correctly matches \process.env.VAR || 'default'\ patterns
• No false positive/negative risks identified
• Exit codes (0/1) correct for CI usage
• Ran locally — all 4 checks pass with ✅

8. CI workflow ✅
actions/checkout@v6 is valid (latest v6.0.2). Path triggers correctly scoped. workflow_dispatch included for manual runs.

Pre-existing issue (not introduced by this PR):
Bicep output \AZURE_OPENAI_API_EMBEDDINGS_DEPLOYMENT_NAME\ is set to \�mbeddingApiVersion\ — should likely be \�mbeddingModelName. Consider a follow-up fix.

[diberry]

Donna (Tester) — ✅ Approved

Test Results

1. Validation Script (scripts/validate-config.mjs)

• ✅ Happy path:
  ode scripts/validate-config.mjs\ exits 0 with all 4 checks passing (GPT model, GPT API version, embedding model, embedding API version)
• ✅ npm script:
  pm run validate:config\ works correctly via package.json
• ✅ Mismatch detection: Intentionally broke sample.env → script correctly detected mismatch, printed \❌ MISMATCH, exited with code 1
• ✅ Source fallback checks: Both \packages-v1/.../llm.ts\ and \packages/.../llm.ts\ fallbacks validated against Bicep

2. Edge Cases

• ⚠️ Missing sample.env: Script crashes with unhandled ENOENT (no try-catch around \parseSampleEnv). Non-blocking — file will always exist in repo, but a friendlier error message would be nice.
• ⚠️ Missing main.bicep: Same crash behavior. Non-blocking for same reason.
• ✅ Missing source files: Handled gracefully (try-catch exists for individual source file reads, logs warning and continues)

3. CI Workflow

• ✅ \�ctions/checkout@v6\ confirmed as latest version (v6.0.2)
• ✅ \�ctions/setup-node@v4\ with Node 20 — correct
• ✅ Trigger paths cover: sample.env, infra/**, packages source configs, the script itself
• ✅ \workflow_dispatch\ allows manual runs
• ✅ \permissions: contents: read\ — minimal and correct

4. Build Verification

• ✅
  pm install\ — 388 packages installed (12 pre-existing vulns, unrelated to this PR)
• ✅
  pm run build\ — tsc compiles cleanly for both workspaces, zero errors

5. Regression Risk Assessment

• maxTokens 100→1000: Bug fix. 100 was truncating responses.
• gpt-4o→gpt-4.1-mini: Safe. Env vars override defaults; only affects new deployments.
• text-embedding-ada-002→text-embedding-3-small: ⚠️ Note: existing vector stores built with ada-002 would need re-embedding. Acceptable for a sample repo upgrade.
• API version 2024-10-21→2025-01-01-preview: Safe with env var override pattern.

Summary

All config values are consistent across sample.env ↔ Bicep ↔ source fallbacks. Validation script works correctly for its intended purpose. Build is clean. Minor suggestion: add try-catch around primary file reads for friendlier CI error messages, but not blocking.