Skip to content

Isolate Google ADK's deterministic providers on a workflow-private random stream - #1854

Merged
tconley1428 merged 30 commits into
mainfrom
fix/adk-providers-in-workflow-threads
Oct 5, 2026
Merged

tconley1428 merged 30 commits into
mainfrom
fix/adk-providers-in-workflow-threads

Conversation

@DABH

@DABH DABH commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

What was changed

1.34.0 (via #1675) made GoogleAdkPlugin install ADK's time, uuid, and random providers as ContextVar defaults, so they finally apply inside workflow tasks. This PR finishes that work:

  • ADK ids and retry jitter draw from a workflow-private deterministic stream (a workflow.new_random("temporalio.contrib.google_adk_agents") per run, kept in a WeakKeyDictionary keyed by the SDK's per-run runtime object so it works during __init__ and with __slots__ classes) instead of workflow.random(). new_random() gains an optional name mixed into the seed: an unnamed instance starts out identical to workflow.random(), which would have made the Nth ADK id equal the Nth workflow.uuid4() (Brian's RequestInput example). Without a name nothing changes. ADK's draw count no longer shifts the sequences user code sees from workflow.random() / workflow.uuid4(), including across google-adk upgrades that change how many ids ADK generates. The plugin builds its v4 UUIDs from that stream itself, with the same construction as workflow.uuid4(); there are no public API changes.
  • Read-only contexts (query handlers, update validators) keep the workflow's deterministic time and get fresh entropy without touching workflow state, the same rule the OpenTelemetry id generator uses. Those contexts are never replayed, and fresh entropy is the only option there since workflow.random() raises and new_random() needs a reseed callback read-only mode forbids. In 1.34.0 an ADK id or random draw inside a query fails it with While in read-only function, action attempted: random.
  • _install_provider also rebinds ADK's _default_* bindings, so reset_*_provider() restores the Temporal providers instead of the standard-library ones (in 1.34.0 a reset silently returns ADK to wall-clock time). Installation takes a lock, is idempotent, and warns when it replaces an override set before the worker started.

💥 Breaking change

Workflows started on 1.34.0 that generated ADK ids or jitter — e.g. one parked on a HITL response whose recorded function-call id was minted from the shared stream — may not replay deterministically across this upgrade. 1.34.0 carried the same caveat for the previous transition; the module is experimental and 1.34.0 shipped on 2026-09-30, so the window of affected workflows is as small as it will ever be. Drain such workflows or use worker versioning. Logged under Breaking Changes.

Testing

tests/contrib/google_adk_agents/test_adk_platform_providers.py: 14 tests over sandboxed and unsandboxed workers plus replay — provider visibility from fresh threads, read-only draws, reset semantics, idempotence, the override warning, outside-workflow fallbacks, and ids drawn in __init__ and in a __slots__ workflow. Run against 1.34.0's plugin, 8 of them fail on behavior (the read-only query failure above, wall-clock time after reset_time_provider(), non-idempotent re-install, no warning on an orphaned override), 3 fail only because they reference this PR's module-level helpers, and 1 passes. Full ADK and LangSmith suites and the lint gate pass on the merged tree.

ADK keeps its time, id, and random providers in contextvars.ContextVars.
GoogleAdkPlugin set them in the worker's context, but workflow tasks run
on the workflow task executor's threads, which start with an empty
context, so ADK code inside a workflow read the defaults: wall-clock time
and uuid.uuid4() for session, event, invocation, and function-call ids.
Only debug mode, which runs activations inline, saw the deterministic
values.

Rebind each google.adk.platform ContextVar to one whose default is the
Temporal provider so it is visible from every context, on Worker and
Replayer alike. Also install the random provider ADK added in 2.8.0 and
raise the google-adk floor to 2.8.0.
@DABH
DABH requested review from a team as code owners September 11, 2026 06:07
Use workflow.time() for the time provider, warn when installing replaces
a provider set earlier in the calling context, and document that
overrides must be made after the worker starts and that ADK id and
random generation raise ReadOnlyContextError in read-only contexts.
Tests assert provider identity.
@DABH

DABH commented Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Notes for review:

  • uv.lock: besides google-adk 2.4.0 -> 2.8.0, the diff carries google-genai 2.11.0 -> 2.20.0 (ADK 2.8.0 requires google-genai>=2.19), a new aiohttp edge, and marker/exclude-newer re-serialization from a newer uv. uv lock --check passes.
  • Behaviour documented in the CHANGELOG and README: installation warns if it replaces an ADK provider set earlier in the calling context (overrides must be made after the worker starts or from workflow code), and, like workflow.uuid4(), ADK id generation and get_random() in query handlers and update validators now raise ReadOnlyContextError instead of returning a random value.
  • The README's "Support for custom span exporters" bullet is removed as a stale-doc cleanup, unrelated to the fix.
  • Validation: uv run pytest tests/contrib/google_adk_agents -> 58 passed, 5 skipped (tests needing GOOGLE_API_KEY and the CI-skipped MCP tests); uv run poe lint (ruff, pyright, mypy, basedpyright, pydocstyle) passes.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

ADK’s public reset functions can restore nondeterministic standard-library providers inside workflows.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Ensures Google ADK uses Temporal’s deterministic time, UUID, and random providers across worker and replay threads.

Changes:

  • Installs thread-visible, idempotent ADK provider defaults.
  • Adds worker, sandbox, replay, fallback, and override tests.
  • Requires Google ADK 2.8.0 and documents compatibility implications.
File summaries
File Description
temporalio/contrib/google_adk_agents/_plugin.py Implements deterministic provider installation.
tests/contrib/google_adk_agents/test_adk_platform_providers.py Tests provider behavior across execution contexts.
temporalio/contrib/google_adk_agents/README.md Documents provider semantics.
pyproject.toml Raises the Google ADK minimum version.
uv.lock Locks Google ADK 2.8.0 and dependencies.
CHANGELOG.md Records the fix and replay compatibility warning.
Review details
  • Files reviewed: 5/6 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread temporalio/contrib/google_adk_agents/_plugin.py
Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
reset_*_provider() restores the module's _default_* binding, so install
now rebinds that too; a set-then-reset cycle lands back on the
deterministic providers instead of wall clock and stdlib random. ADK ids
and randoms now come from a workflow.new_random() cached on the workflow
instance (as the opentelemetry and langsmith integrations do) so ADK
draws never shift the sequence user code sees from workflow.random(),
with an explicit read-only guard so query handlers cannot advance the
cached stream.
@DABH DABH closed this Sep 14, 2026
@DABH DABH reopened this Sep 14, 2026
Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
Per review: query handlers and update validators now get a
nondeterministic fallback random instead of ReadOnlyContextError - their
results are never replayed, and the guard only has to keep the cached
private stream untouched (a replay test proves it stays untouched).
workflow.uuid4() accepts an optional keyword-only random argument so the
uuid-from-generator derivation lives in one place; the ADK id provider
and langsmith's _uuid_from_random now delegate to it. Changelog entries
rehomed under Unreleased after the 1.33.0 cut and reworded for the
private-stream design.
Comment thread temporalio/workflow/_context.py Outdated
Comment thread temporalio/contrib/google_adk_agents/_plugin.py
DABH and others added 3 commits September 15, 2026 11:43
Query handlers and update validators now get time.time() from the ADK
time provider: their results are never replayed, and workflow.time()
would hand them the last activation's timestamp, stale by however long
the workflow has been parked. Covered by a query in the
query-during-run test.
Per review: the keyword random shadowed the module-level random()
function inside uuid4, which forced the body to inline the runtime
call. With the rename the body simply calls random() again, and
callers read as workflow.uuid4(rng=...).
Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
DABH and others added 7 commits September 17, 2026 11:42
Per review: the read-only and outside-workflow fallbacks now return a
fresh random.Random() instead of module-level singletons. Nothing needs
to persist there - read-only results are never replayed, and ADK's
guidance to return an existing instance only matters for a seeded
generator whose sequence must continue; an unseeded one draws fresh OS
entropy either way, and ADK's only get_random() caller uses the value
immediately for retry jitter.
…workflow-threads

Resolve temporalio/contrib/google_adk_agents/_plugin.py in favor of this
branch's provider installation (module-level providers, the locked
_install_provider that also rebinds ADK's _default_* bindings, read-only
handling, and the workflow-private random stream), which supersedes the
three-argument _install_provider and the workflow.random()-sharing
providers added by #1675; keep #1675's optional model SDK passthrough.

Reconcile the CHANGELOG to one entry for the google-adk>=2.8.0 bump and
drop #1675's note about workflow.random()/workflow.uuid4() sequences
shifting across the upgrade, which no longer applies now that ADK draws
from a private stream. Reword three #1675 test comments to match.
…workflow-threads

Resolve _plugin.py by keeping both the OpenTelemetry passthrough bullet from
#1899 and this branch's providers bullet, take main's uv.lock (pyproject is
unchanged by this branch), and rebuild the CHANGELOG from main's text: the
union merge had moved this branch's notes into the released 1.34.0 section.
The Unreleased entries now describe this branch relative to 1.34.0, which
already shipped #1675's provider installation, including a breaking-change
note for workflows started on 1.34.0.
@DABH DABH changed the title Apply Google ADK deterministic providers inside workflow tasks Isolate Google ADK's deterministic providers on a workflow-private random stream Oct 1, 2026
@DABH
DABH requested a balanced review from Copilot October 1, 2026 03:47

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The read-only fallback also affects replayed dynamic workflow configuration, introducing nondeterministic time and entropy there.

Review effort: Balanced
Findings: 1 High severity · 3 Low severity

Open (4)
Resolved since last review (1)

Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread temporalio/workflow/_context.py Outdated
Comment thread tests/test_workflow.py Outdated
DABH added 2 commits October 1, 2026 00:25
is_read_only() is also true while a dynamic workflow's dynamic_config runs,
and that hook is replayed, so a wall-clock fallback there was not replay
safe. The time provider now returns workflow.time() in every in-workflow
context; a query sees the activation timestamp instead of the wall clock,
which is harmless because nothing a query computes is persisted. Ids and
randoms keep the fresh-entropy fallback in read-only contexts, where no
deterministic source is available (workflow.random() raises and new_random()
needs a reseed callback that read-only mode forbids); the comments now say so
and name dynamic_config as the one replayed read-only hook.

Also stop claiming uuid4(rng=...) touches no workflow state: it advances the
supplied generator, which a caller could make the shared one.
@DABH
DABH enabled auto-merge (squash) October 1, 2026 05:47
Comment thread temporalio/workflow/_context.py Outdated
DABH and others added 4 commits October 1, 2026 12:54
With an explicit generator uuid4 reads no workflow state, so it does not
belong under workflow.*. The plugin now builds the UUID from its private
stream with the same construction, and contrib.langsmith goes back to its
own one-line construction.

@brianstrauch brianstrauch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes for two reproduced issues in the ADK RNG provider: failures during workflow construction or with slotted classes, and nondeterministic IDs from replayed dynamic_config callbacks.

The existing ADK suite (88 passed, 5 skipped) and poe lint passed on this head. Focused real-worker reproductions exposed the cases described below.

Comment thread temporalio/contrib/google_adk_agents/_plugin.py Outdated
Comment thread temporalio/contrib/google_adk_agents/_plugin.py
DABH added 2 commits October 1, 2026 21:04
…-only draws to queries and validators

workflow.instance() is None during __init__, and a class with __slots__
cannot take a cache attribute, so storing the private stream on the
workflow object failed in both cases. Key it by the SDK's per-run runtime
object in a WeakKeyDictionary instead.

The entropy fallback for read-only code also covered a dynamic workflow's
dynamic_config, which is replayed; ids drawn there differed on replay. Only
query handlers and update validators are never replayed, so only they get
fresh entropy. Other read-only contexts raise ReadOnlyContextError, as
workflow.random() does. The runtime exposes the internal distinction as
workflow_in_query_or_validator().
@DABH
DABH requested a review from brianstrauch October 2, 2026 02:07
DABH added 2 commits October 2, 2026 12:47
workflow.new_random() seeds from the same value as workflow.random(), so
an unnamed private stream starts out identical to the workflow's own: the
Nth ADK id equaled the Nth workflow.uuid4(). new_random() now takes an
optional name that is mixed into the seed (and into every reseed), and
the plugin uses one. Without a name nothing changes.
…enerator does

Drop the workflow_in_query_or_validator runtime accessor and the
ReadOnlyContextError for replayed read-only callbacks: minting ADK ids
inside dynamic_config or a patch activation callback is not worth
runtime plumbing. Read-only code gets a fresh unseeded generator and the
private stream is never touched there, matching contrib.opentelemetry.
@tconley1428
tconley1428 disabled auto-merge October 5, 2026 16:25
@tconley1428
tconley1428 merged commit 049ebd6 into main Oct 5, 2026
33 of 34 checks passed
@tconley1428
tconley1428 deleted the fix/adk-providers-in-workflow-threads branch October 5, 2026 16:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants