Skip to content

launch: make the three-source, bring-your-own-key scenario a one-shot - #31

Closed
vladisemenov wants to merge 1 commit into
spiceai:trunkfrom
vladisemenov:improve-launch-one-shot
Closed

vladisemenov wants to merge 1 commit into
spiceai:trunkfrom
vladisemenov:improve-launch-one-shot

Conversation

@vladisemenov

Copy link
Copy Markdown

Summary

Fixes and additions from running the launch skill end to end on a Snowflake, Databricks, and Postgres scenario, with stand-in data and LLM keys that did not exist yet. The scenario deployed and verified on Spice.ai Cloud (v2.3.2): 12/12 checks, five datasets and two cross-source views returning rows, an MCP session with 8 sql calls and no session_not_found, p50 555 ms and p99 636 ms over 50 uncached queries.

Changes

  • local no longer reports a healthy spicepod as not ready. It stopped when the datasets settled, about 15 s in, but views are not listed by the runtime and a join over a 1.5M-row table took about 32 s to initialize. It now waits for /v1/ready and still stops early when a component reports Error.
    • Before: the same spicepod gave "ok": false, "ready": false, "error": "the spicepod did not become ready locally".
    • After: "ok": true, "ready": true.
    • A manual run of the spicepod reached /v1/ready after about 32 s and returned 150,000 rows from customer_orders and 25 from country_overview.
  • Credential hint. After spice cloud login reported success, preflight still returned management_token_missing. The hint and Step 3 now give the path that worked: a personal access token in .env.local, entered in an editor and not pasted into a command (a silent read -s prompt took no input in an embedded terminal). They also tell the agent not to search for the credential itself.
  • SKILL.md. Ask all missing questions in one message, including the Cloud credential. Interpret "users bring their own LLM key" explicitly: hosted models with owner-supplied keys, or a client-side LLM over MCP. Spice has no per-request key. Keep a model out of the spicepod until its key is stored, because a missing key blocks readiness for the whole deploy. Put the project directory outside any Git checkout. Add a note on sharing with a teammate (a Spice.ai username, not an email; access is org-wide; confirm the role). Two new Don'ts.
  • scenarios.md. A Databricks section checked against the connector docs, multi-provider model guidance (Anthropic and xAI are Alpha), and a TPC-H layout that gives Snowflake, Databricks, and Postgres each a role.
  • examples/spicepod.unified-data.yaml. That layout, with the Claude, OpenAI, and Grok blocks commented out. spice validate passes with the models commented and uncommented (3 models), and the same layout was deployed and verified as above.
  • Eval 4. The scenario in generic terms. Its assertion regexes match the spicepod that was deployed.

Verification

  • make check test-distribution release-preview passes (13 tests, 17 skills, no problems).
  • SKILL.md is 372 lines (cap 500); the skill description is unchanged.
  • The staged diff was scanned for personal, internal, and credential strings; none found.

Not included

  • No version bump, since versions follow Spice releases.
  • The full skill was not re-run through an agent against eval 4; only the helper fix, the example, and the eval regexes were exercised.

Fixes and additions found by running the skill end to end on a Snowflake,
Databricks, and Postgres scenario with stand-in data and pending LLM keys.

- local: wait for /v1/ready instead of stopping when the datasets settle.
  Views are not listed by the runtime and a join over a 1.5M-row table took
  about 32 s, so the old check reported "did not become ready locally" for a
  spicepod that was healthy. It still stops early when a component errors.
- preflight: the credential hint now covers the case where `spice cloud
  login` succeeded but the helper finds no credential: a personal access
  token in .env.local, entered in an editor, not pasted into a command.
- SKILL.md: ask every missing question in one message, including the Cloud
  credential; interpret "users bring their own LLM key" explicitly (hosted
  models versus a client-side LLM over MCP; Spice has no per-request key);
  keep a model out of the spicepod until its key is stored; put the project
  directory outside any Git checkout; add a teammate-sharing note (Spice.ai
  username not email, org-wide access, confirm the role); two new Don'ts.
- scenarios.md: Databricks connector section checked against the connector
  docs, multi-provider model guidance (Anthropic and xAI are Alpha), and the
  TPC-H layout that gives Snowflake, Databricks, and Postgres each a role.
- examples/spicepod.unified-data.yaml: that layout, validated with and
  without the model blocks, deployed and verified on Spice.ai Cloud.
- evals: add the scenario as eval 4.

make check, test-distribution, and release-preview pass.
@vladisemenov

Copy link
Copy Markdown
Author

@lukekim please review I have read only permissions

@lukekim lukekim closed this Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants