Skip to content

Gemini Decision 1 as a decision model; models start on the current SCR base image - #86

Merged
Criptic merged 3 commits into
mainfrom
feature/gemini-decision
Nov 12, 2026
Merged

Criptic merged 3 commits into
mainfrom
feature/gemini-decision

Conversation

@Criptic

@Criptic Criptic commented Nov 12, 2026

Copy link
Copy Markdown
Member

Summary

Google released Gemini Decision 1, a decision (System One) model behind its own Decisions API. This adds it as a decision model that the SCR container calls directly at Google, and fixes two defects found on the way.

  • Gemini Decision 1 through Google's API. Google's Decisions API takes the System One questions and returns the System One answers. The request differs in three ways: the state goes as input, the key in the x-goog-api-key header, and the token counts are named total_input_tokens/total_output_tokens.
    • The new dec_gemini score template translates inside the SCR container, so the container keeps the System One contract. The Prompt Builder and SAS Intelligent Decisioning need no change.
    • A JSON state is sent as text: Google takes an object there for a content block and refuses it.
    • mdb's google adapter serves decision models: mdb add google gemini-decision-1 --kind decision finds the model in Google's live list (createDecision), and mdb validate --live makes a decision call.
    • New definition Decision-Definitions/gemini_decision_1 with the Google key entry and its fact-sheet row. Google's and OpenRouter's catalog snapshots list the model; OpenRouter serves it on its System One endpoint with the unchanged template.
  • Fixed: a decision model's answer and confidence answer the first question. The scorers took the first answer in the provider's order. Gemini Decision 1 returns the answers in another order than the questions were asked, via Google and via OpenRouter alike, so the headline answered the wrong question in 4 of 5 calls. The shared partials/_decision_answers.py.j2 picks the first question asked; every decision scorer is regenerated. Jev keeps the order, so it was not affected.
  • Fixed: models start on the current SCR base image. The SCR base image now ships pandas 3.1.1 for Python 3.11, built for numpy 2, and SCR's own server imports pandas at start. Our images pinned numpy==1.26.4, so a freshly published container exited with "C extension: None not built" before it loaded the scorer. Every republished model would have hit this.
    • API models install requests only; no API scorer uses numpy.
    • Open-weight profiles (hf-transformers, hf-sentence-transformers, hf-onnx) and the hand-kept requirements of Llama 3.2, Mistral Nemo and Phi-3/3.5 require the numpy the base image already has (KEEP_BASE_NUMPY). An install that needs another numpy fails at publish time, not in the running container.
    • _Base_Definition installs requests and tiktoken only.

Validated on viya4-s2 (2026-11-12)

  • gemini_decision_1: registered, published to MyPrivateDocker and deployed in namespace llm.
    • Direct calls: the headline was right in every call, also when Google reordered the answers. A JSON state worked, a call without a key returned SCR's 422 naming API_KEY, and a call took about 0.3 s in the container.
    • Prompt Builder: a decision run beside Jev and Von matched all expected answers. 271 tokens, 0.35 s, estimated cost $0.000027; the key came from the Google entry of the credential domain.
  • Open-weight images: copies of qwen_25_05b and all_minilm_l6_v2 under their own ids were built on the current base image. Both kept numpy 2.4.6 with pandas 3.1.1, started, and answered (Qwen "OK" in 0.42 s, an embedding of 11 tokens). The copies were removed afterwards. Von and the hand-kept definitions were not rebuilt.

Tests

  • New tests for the dec_gemini template (request shape, header, headline order, usage, MAS-style inputs), the Google adapter's catalog and smoke test, the importer, the headline order of the System One template, and the numpy handling of every profile and every committed requirements.json.
  • CI matrix on Python 3.11 and 3.14: mdb 2716 passed, rag 1562, logging 1410, setup 465, po 14. mdb generate --all --check, mdb validate --all and mdb package-check pass.

Release note

  • mdb package-check --release reports the new row of Data/decision_fact_sheet.csv until the next package export, beside the enable_weaviate drift that RAG Builder: Weaviate as a vector store #84 left.
  • A deployed model picks up the fixes only after mdb register --update and a republish.

🤖 Generated with Claude Code

Criptic and others added 3 commits November 12, 2026 11:40
…ated inside the SCR container

Google's Decisions API takes the System One questions and returns the
System One answers, but wants the state as `input`, the key in the
x-goog-api-key header, and names its token counts total_input_tokens and
total_output_tokens. The new dec_gemini score template translates, so the
container keeps the System One contract for the Prompt Builder and SAS
Intelligent Decisioning. A JSON state is sent as text: Google takes an
object there for a content block and refuses it.

- google adapter: decision_template dec_gemini, endpoint
  /v1beta/decisions, createDecision models listed as kind decision,
  a decision smoke test
- Decision-Definitions/gemini_decision_1 (`Google` key entry), its fact
  sheet row; Google's and OpenRouter's snapshots list the model
- importer: recognises a dec_gemini scorer

Fixed: `answer` and `confidence` took the first answer in the provider's
order. Gemini Decision 1 returns the answers in another order than the
questions were asked (via Google and via OpenRouter), so the headline
answered the wrong question in 4 of 5 calls. They now answer the first
question asked (partials/_decision_answers.py.j2, used by dec_systemone_api,
dec_von and dec_gemini); every decision scorer is regenerated.

Live, 2026-11-12: mdb validate --live, mdb test (5 runs) and --mas pass;
a JSON state, MAS-style inputs and a bad key behave as expected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Weik <geekupyourlife@gmail.com>
gemini_decision_1 published on zeus (2026-11-12) did not start: the
container exited with code 3 and "C extension: None not built" before it
loaded the scorer. The SCR base image now ships pandas 3.1.1 for Python
3.11, built for numpy 2, and SCR's own server imports pandas at start; the
api-wrapper profile's numpy==1.26.4 pin downgraded numpy under it. Jev's
image of 2026-10-20 still had pandas 3.0.3, which takes numpy 1.26.

The api-wrapper profile installs requests only - no scorer of an API
model uses numpy. Every requirements.json of an API model is regenerated.
Republished on zeus, the container starts and answers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Weik <geekupyourlife@gmail.com>
The hf-transformers and hf-sentence-transformers profiles pinned
numpy==1.26.4 as well, and so did the hand-kept requirements of
llama_32_1b, llama_32_3b, mistral_nemo, phi_35_mini, phi_3_mini_4k and the
_Base_Definition: on the base image of 2026-11 (pandas 3.1.1 for Python
3.11, built for numpy 2) every republished one would exit at start like
gemini_decision_1 did.

An install that could move numpy now requires the numpy the image already
has (KEEP_BASE_NUMPY: "numpy==$(python3 -c '...numpy.__version__')"), on
an old base image as on the new one; an install that needs another numpy
fails at publish time instead of in the running container. The
_Base_Definition installs requests and tiktoken, which need no numpy.

Built on zeus 2026-11-12 from copies with their own ids
(qwen_25_05b_check, all_minilm_l6_v2_check; not committed): both images
kept numpy 2.4.6 with pandas 3.1.1, started, and answered - Qwen "OK" in
0.42 s, all-MiniLM-L6-v2 an embedding of 11 tokens.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: David Weik <geekupyourlife@gmail.com>
@Criptic
Criptic merged commit eac601d into main Nov 12, 2026
3 checks passed
@Criptic
Criptic deleted the feature/gemini-decision branch November 12, 2026 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant