Repository navigation
AgentCore Support (1/3) - Add AgentCore provider references #25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
justinschoeff
wants to merge
3
commits into
main
Choose a base branch
from
agentcore/references
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
99 changes: 99 additions & 0 deletions
99
assets/temporal-cloud-serverless-worker-agentcore-role.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # CloudFormation template for creating an IAM role that Temporal Cloud can assume to invoke AgentCore runtimes. | ||
| AWSTemplateFormatVersion: "2010-09-09" | ||
| Description: Creates an IAM role that Temporal Cloud can assume to invoke Amazon Bedrock AgentCore runtimes for Serverless Workers. | ||
|
|
||
| Parameters: | ||
| AssumeRoleExternalId: | ||
| Type: String | ||
| Description: The External ID provided by Temporal Cloud | ||
| AllowedPattern: "[a-zA-Z0-9_+=,.@-]*" | ||
| MinLength: 5 | ||
| MaxLength: 45 | ||
|
|
||
| AgentRuntimeARNs: | ||
| Type: CommaDelimitedList | ||
| Description: >- | ||
| Comma-separated list of AgentCore runtime ARNs to invoke (e.g., | ||
| arn:aws:bedrock-agentcore:us-west-2:123456789012:runtime/temporal_worker-XXXXXXXXXX). | ||
| Append a wildcard (.../runtime/temporal_worker-XXXXXXXXXX*) to cover the | ||
| runtime's endpoints as well, which GetAgentRuntimeEndpoint is scoped to. | ||
|
|
||
| RoleName: | ||
| Type: String | ||
| Default: "Temporal-Cloud-Serverless-Worker" | ||
|
|
||
| Metadata: | ||
| AWS::CloudFormation::Interface: | ||
| ParameterGroups: | ||
| - Label: | ||
| default: "Temporal Cloud Configuration" | ||
| Parameters: | ||
| - AssumeRoleExternalId | ||
| - Label: | ||
| default: "AgentCore Configuration" | ||
| Parameters: | ||
| - AgentRuntimeARNs | ||
| - RoleName | ||
| ParameterLabels: | ||
| AssumeRoleExternalId: | ||
| default: "External ID (provided by Temporal Cloud)" | ||
| AgentRuntimeARNs: | ||
| default: "AgentCore Runtime ARNs (comma-separated list)" | ||
| RoleName: | ||
| default: "IAM Role Name" | ||
|
|
||
| Resources: | ||
| TemporalCloudServerlessWorker: | ||
| Type: AWS::IAM::Role | ||
| Properties: | ||
| RoleName: !Sub "${RoleName}-${AWS::StackName}" | ||
| AssumeRolePolicyDocument: | ||
| Version: "2012-10-17" | ||
| Statement: | ||
| - Effect: Allow | ||
| Principal: | ||
| AWS: | ||
| [ | ||
| arn:aws:iam::902542641901:role/wci-lambda-invoke, | ||
| arn:aws:iam::160190466495:role/wci-lambda-invoke, | ||
| arn:aws:iam::819232936619:role/wci-lambda-invoke, | ||
| arn:aws:iam::829909441867:role/wci-lambda-invoke, | ||
| arn:aws:iam::354116250941:role/wci-lambda-invoke, | ||
| ] | ||
| Action: sts:AssumeRole | ||
| Condition: | ||
| StringEquals: | ||
| "sts:ExternalId": [!Ref AssumeRoleExternalId] | ||
| Description: "The role Temporal Cloud uses to invoke AgentCore runtimes for Serverless Workers" | ||
| MaxSessionDuration: 3600 # 1 hour | ||
|
|
||
| TemporalCloudAgentCoreInvokePermissions: | ||
| Type: AWS::IAM::Policy | ||
| DependsOn: TemporalCloudServerlessWorker | ||
| Properties: | ||
| PolicyName: "Temporal-Cloud-AgentCore-Invoke-Permissions" | ||
| PolicyDocument: | ||
| Version: "2012-10-17" | ||
| Statement: | ||
| - Effect: Allow | ||
| Action: | ||
| - bedrock-agentcore:InvokeAgentRuntime | ||
| - bedrock-agentcore:GetAgentRuntimeEndpoint | ||
| Resource: !Ref AgentRuntimeARNs | ||
| Roles: | ||
| - !Sub "${RoleName}-${AWS::StackName}" | ||
|
|
||
| Outputs: | ||
| RoleARN: | ||
| Description: The ARN of the IAM role created for Temporal Cloud | ||
| Value: !GetAtt TemporalCloudServerlessWorker.Arn | ||
| Export: | ||
| Name: !Sub "${AWS::StackName}-RoleARN" | ||
|
|
||
| RoleName: | ||
| Description: The name of the IAM role | ||
| Value: !Ref RoleName | ||
|
|
||
| AgentRuntimeARNs: | ||
| Description: The AgentCore runtime ARNs that can be invoked | ||
| Value: !Join [", ", !Ref AgentRuntimeARNs] |
68 changes: 68 additions & 0 deletions
68
assets/temporal-self-hosted-serverless-worker-agentcore-role.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # CloudFormation template for creating an IAM role that a self-hosted Temporal Service can assume to invoke AgentCore runtimes. | ||
| AWSTemplateFormatVersion: '2010-09-09' | ||
| Description: | ||
| Creates an IAM role that a self-hosted Temporal Service can assume to invoke Amazon Bedrock AgentCore runtimes. | ||
|
|
||
| Parameters: | ||
| TemporalIamRoleArn: | ||
| Type: String | ||
| Description: The ARN of the IAM role or user that the Temporal Service runs as. | ||
|
|
||
| AssumeRoleExternalId: | ||
| Type: String | ||
| Description: A unique identifier to prevent confused deputy attacks. | ||
| AllowedPattern: '[a-zA-Z0-9_+=,.@-]*' | ||
| MinLength: 5 | ||
| MaxLength: 45 | ||
|
|
||
| AgentRuntimeARNs: | ||
| Type: CommaDelimitedList | ||
| Description: >- | ||
| Comma-separated list of AgentCore Runtime ARNs that Temporal may access. Append a wildcard to each Runtime ARN | ||
| to include its endpoints. | ||
|
|
||
| RoleName: | ||
| Type: String | ||
| Default: 'Temporal-AgentCore-Worker' | ||
|
|
||
| Resources: | ||
| TemporalAgentCoreWorker: | ||
| Type: AWS::IAM::Role | ||
| Properties: | ||
| RoleName: !Ref RoleName | ||
| AssumeRolePolicyDocument: | ||
| Version: '2012-10-17' | ||
| Statement: | ||
| - Effect: Allow | ||
| Principal: | ||
| AWS: [!Ref TemporalIamRoleArn] | ||
| Action: sts:AssumeRole | ||
| Condition: | ||
| StringEquals: | ||
| 'sts:ExternalId': [!Ref AssumeRoleExternalId] | ||
| Description: The role the Temporal Service uses to invoke AgentCore runtimes for Serverless Workers | ||
| MaxSessionDuration: 3600 | ||
|
|
||
| TemporalAgentCoreInvokePermissions: | ||
| Type: AWS::IAM::Policy | ||
| Properties: | ||
| PolicyName: 'Temporal-AgentCore-Invoke-Permissions' | ||
| PolicyDocument: | ||
| Version: '2012-10-17' | ||
| Statement: | ||
| - Effect: Allow | ||
| Action: | ||
| - bedrock-agentcore:InvokeAgentRuntime | ||
| - bedrock-agentcore:GetAgentRuntimeEndpoint | ||
| Resource: !Ref AgentRuntimeARNs | ||
| Roles: | ||
| - !Ref TemporalAgentCoreWorker | ||
|
|
||
| Outputs: | ||
| RoleARN: | ||
| Description: The ARN of the IAM role created for the Temporal Service | ||
| Value: !GetAtt TemporalAgentCoreWorker.Arn | ||
|
|
||
| AgentRuntimeARNs: | ||
| Description: The AgentCore Runtime ARNs that Temporal may access | ||
| Value: !Join [', ', !Ref AgentRuntimeARNs] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| # AgentCore — Execution-model constraints | ||
|
|
||
| These are the consequences of AgentCore Runtime's execution model. **The WCI invokes a named AgentCore Runtime endpoint when it needs capacity. The Runtime starts a standard long-running Worker inside a Runtime session, and that Worker polls until it drains or AgentCore ends its compute.** | ||
|
|
||
| ## Pre-release status | ||
|
|
||
| AgentCore Runtime support is in **Pre-release**, and its APIs may change in backwards-incompatible ways. A Temporal Cloud Namespace needs Pre-release access, which the user requests through a [support ticket](https://docs.temporal.io/evaluate/cloud/support#support-ticket) or their account team. A self-hosted Temporal Service v1.32.0 or later does not need an access request. See `self-hosted.md`. | ||
|
|
||
| The Cloud Namespace must be **hosted on AWS**. | ||
|
|
||
| ## Worker code is a standard Worker inside a Runtime handler | ||
|
|
||
| AgentCore Workers are standard long-lived Temporal Workers. There is no serverless Worker package. The Runtime entry point must implement the AgentCore Runtime HTTP contract by serving `/invocations` and `/ping`. For a Serverless Worker, `/invocations` starts the Worker as background work and acknowledges the request without waiting for the Worker to stop. The per-language entry point is in the selected `sdk-<language>.md` reference. | ||
|
|
||
| ## Compute lifetime | ||
|
|
||
| - **Up to 8 hours per compute.** With the serverless microVM compute type, a Runtime session can run for up to 8 hours. The `maxLifetime` setting accepts 60–28800 seconds for microVM runtimes and defaults to 28800 seconds. Its timer starts when the microVM is created and cannot be reset. | ||
| - **No fixed invocation deadline.** An AgentCore Worker has no invocation deadline to monitor. It polls until it drains or AgentCore ends its compute. | ||
| - **The session is not durable storage.** AgentCore can resume a session on new compute after the previous compute ends, and a later Task can run on another Worker. Keep the state a Workflow needs in the Workflow or another durable store. | ||
| - **Process-local reuse is an optimization only.** The Worker can reuse initialization work, in-memory caches and temporary files while its Runtime compute remains available. Do not make Workflow progress depend on them. | ||
| - **Workflow duration is unbounded.** A Workflow continues across as many Worker processes as needed. Durable Timers and waits for approvals or external events do not keep compute running. | ||
|
|
||
| ## Two independent stop controls | ||
|
|
||
| Two sets of controls determine when the Worker stops. Both must be configured deliberately. | ||
|
|
||
| | Control | Owner | What it measures | | ||
| |---|---|---| | ||
| | Worker idle and graceful-shutdown policy | The Runtime handler you write | When the Worker has been idle. It then stops polling and waits for in-flight Activities. | | ||
| | `idleRuntimeSessionTimeout` | AgentCore lifecycle settings | Time since the last **AgentCore Runtime invocation** to the session. Default 900 seconds. Polling Temporal does **not** reset it. | | ||
| | `maxLifetime` | AgentCore lifecycle settings | Time since the compute was created. Default and microVM maximum: 28800 seconds (8 hours). | | ||
|
|
||
| The idle session timeout is not a Worker idle timer, and it does not replace one. Implement a Worker shutdown policy in the handler: when the idle condition is met, stop polling and drain in-flight Activities before the handler releases the session. Choose the idle period and drain timeout for the workload, and keep both inside `maxLifetime`. | ||
|
|
||
| `idleRuntimeSessionTimeout` must be less than or equal to `maxLifetime`, and both must be at least 60 seconds. A request that breaks these rules fails with `ValidationException`. When AgentCore ends compute, termination can take up to 15 seconds. | ||
|
|
||
| In the Python sample, the Worker drains after `AGENTCORE_DEBOUNCE_SECONDS` without an Activity starting or finishing. The default is `60`. The sample's drain timeout is 120 seconds, and its `agentcore.json` keeps the AgentCore defaults (`idleRuntimeSessionTimeout: 900`, `maxLifetime: 28800`). These are configured values, not platform limits. | ||
|
|
||
| ## What bounds an Activity: AgentCore ending the compute | ||
|
|
||
| AgentCore can end the compute before an Activity completes. Configure Activity timeouts. For long-running Activities, also configure [Activity Heartbeats](https://docs.temporal.io/encyclopedia/detecting-activity-failures#activity-heartbeat) so a retry can recover the work. A Worker crash follows standard retry semantics: the Activity Timeout fires and Temporal retries the Activity on another Worker. | ||
|
|
||
| ## Autoscaling | ||
|
|
||
| Autoscaling is event-driven (`no-sync`). The WCI invokes individual Runtime sessions when it needs more capacity. It does not manage a target-sized pool. For the shared WCI lifecycle and inspection commands, see [Worker Controller Instance (WCI)](../wci.md). | ||
|
|
||
| When the provider's capacity limit is reached, the WCI cannot add Workers. Tasks wait in the backlog without data loss. | ||
|
|
||
| ## One named endpoint per Worker Deployment Version | ||
|
|
||
| The compute configuration names a Runtime **endpoint** ARN, not a Runtime ARN. Each Worker Deployment Version needs its own named endpoint pinned to one immutable Runtime version. Never configure the `DEFAULT` endpoint, because it moves to the latest Runtime version on every update. → `versioning.md`. | ||
|
|
||
| ## Region and high availability | ||
|
|
||
| The AWS account must be in an [AgentCore-supported Region](https://docs.aws.amazon.com/bedrock-agentcore/latest/devguide/agentcore-regions.html). Compute provider configuration is scoped to one region. On failover of a replicated Namespace, the WCI keeps invoking Workers in the original region until the compute provider is repointed manually. | ||
|
|
||
| ## AgentCore services are not configured by the compute provider | ||
|
|
||
| Configuring AgentCore Runtime as a compute provider does not set up AgentCore Identity, Gateway, Policy, Memory or Observability. Call those services from Activities, and keep credentials out of Workflow state. Writing the agent itself (Workflow and Activity boundaries, Strands, model calls) is out of scope for this skill. Link to [Build a durable agent on Amazon Bedrock AgentCore](https://docs.temporal.io/guides/durable-agent-on-agentcore) instead. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| # AgentCore — Diagnostics and troubleshooting | ||
|
|
||
| Temporal has no AgentCore troubleshooting page. This guide uses only the gotchas documented in the sources above and the shared [Worker Controller Instance (WCI)](../wci.md) inspection flow. If a symptom is not covered here, gather the evidence below and report it to the user. Do not attach an undocumented explanation to it. | ||
|
|
||
| ## Invocation flow (when working correctly) | ||
|
|
||
| 1. Creating the Worker Deployment Version starts its WCI, which invokes the named endpoint and waits for the Worker to register. | ||
| 2. The Runtime's `/invocations` handler starts the Worker in the background and acknowledges. | ||
| 3. The Worker polls the Task Queue as `<deployment-name>/<build-id>`, and the version binds the Task Queue. | ||
| 4. When Tasks arrive and no Worker is polling, the WCI invokes the endpoint again. | ||
| 5. The Worker drains when its idle policy fires, or stops when AgentCore ends the compute. | ||
|
|
||
| ## Start here: did the expected Worker register? | ||
|
|
||
| ```bash | ||
| temporal worker deployment describe-version \ | ||
| --namespace <NAMESPACE> --deployment-name <DEPLOYMENT_NAME> --build-id <BUILD_ID> \ | ||
| --report-task-queue-stats -o json | ||
| ``` | ||
|
|
||
| The decisive signal is that the expected Task Queue appears in `taskQueuesInfos` with the types the Worker polls. That only happens after a Worker that identifies as this deployment name and Build ID has polled. A successful Validate Connection or a Running WCI supports the diagnosis but does not replace this check. Wait for this state to appear rather than for a fixed time. Re-run the command while you wait. If the Task Queue is still not bound after several minutes, stop waiting and use the table below. A bound Task Queue does not show that the version is current or that Tasks complete. | ||
|
|
||
| | Result | Next check | | ||
| |---|---| | ||
| | Task Queue bound | Registration succeeded. Confirm the version is current, then go to [Tasks not completing](#worker-runs-but-tasks-do-not-complete). | | ||
| | Not bound, and the WCI history shows invocation failures | [The Runtime is not invoked](#the-runtime-is-not-invoked) | | ||
| | Not bound, invocation succeeded | [Invoked, but no Worker registers](#invoked-but-no-worker-registers) | | ||
|
|
||
| ## The Runtime is not invoked | ||
|
|
||
| 1. **Validate Connection (Cloud).** In the version's **Actions → Validate Connection**, a failure means Temporal cannot assume the invocation role, get the named endpoint, or invoke the Runtime. Check the following: | ||
| - `--aws-agentcore-endpoint-arn` is the **endpoint** ARN (`…/runtime-endpoint/<name>`), not the Runtime ARN. | ||
| - The role's `AgentRuntimeARNs` is the Runtime ARN **with a trailing `*`**. Without the wildcard, `GetAgentRuntimeEndpoint` on the endpoint is not covered. | ||
| - The External ID is identical in the stack and in the version. | ||
| - Self-hosted: the server's AWS identity is the stack's `TemporalIamRoleArn` and has `sts:AssumeRole` on the role. The server can also reach the AgentCore control-plane and data-plane APIs. → `self-hosted.md`. | ||
| 2. **Stack never finished.** If the role stack failed, check the role-name length. On Cloud, `<ROLE_NAME>-<STACK_NAME>` must be 64 characters or fewer. → `iam.md`. | ||
| 3. **Endpoint not ready.** Check the endpoint's state with `agentcore status --type runtime-endpoint --json`. `CREATE_FAILED` and `UPDATE_FAILED` mean a failure caused by permissions, the artifact, or another configuration problem. The sources document no cause for an endpoint that stays `CREATING` or `UPDATING`. If it is still in one of those states after several minutes, report the state and the `agentcore deploy` output to the user. Do not redeploy to get it to move. | ||
| 4. **Version not current.** If registration succeeded but new Workflows never reach the Worker, set the version current (`setup.md` Step 6). | ||
| 5. **Read the WCI history** for the failing Activity result. Use the inspection commands and interpretation rules in [`../wci.md`](../wci.md#inspect-the-wci). | ||
|
|
||
| ## Invoked, but no Worker registers | ||
|
|
||
| Read the Runtime logs first: | ||
|
|
||
| ```bash | ||
| aws logs tail /aws/bedrock-agentcore/runtimes/<RUNTIME_ID>-<ENDPOINT_NAME> --since 1h --region <AWS_REGION> | ||
| ``` | ||
|
|
||
| - **Name mismatch.** The Runtime's `TEMPORAL_DEPLOYMENT_NAME` / `TEMPORAL_BUILD_ID` differ from the version's. Temporal starts compute that registers as some other version. Fix the environment, which creates a new Runtime version. Then follow `versioning.md`. | ||
| - **Missing versioning settings.** The sample Worker refuses to start without `TEMPORAL_DEPLOYMENT_NAME` and `TEMPORAL_BUILD_ID`. | ||
| - **Cannot reach Temporal.** With a VPC network mode, the VPC needs outbound access to the Temporal Service. For self-hosted, see `self-hosted.md` step 1. | ||
| - **Endpoint runs other code.** The version points at `DEFAULT`, or at a named endpoint that was moved to another Runtime version. → `versioning.md`. | ||
| - **Deploy failed with `uv install failed ... with exit code null`.** `uv` is missing on the operator's machine. | ||
|
|
||
| ## Worker runs but Tasks do not complete | ||
|
|
||
| - **Non-determinism errors after a deploy.** Code changed underneath a live Worker Deployment Version, because the version uses `DEFAULT` or a repointed endpoint. Pinned Workflows are affected too. Restore the old endpoint-to-version mapping, as described in `versioning.md`. | ||
| - **Old code still running after a deploy.** Existing sessions keep the code that was deployed when their microVM was created, until they terminate. | ||
| - **Activities interrupted.** AgentCore ended the compute (`maxLifetime`) before the Activity completed, or the idle policy drained too early. Configure Activity timeouts and Heartbeats so retries recover the work. → `constraints.md`. | ||
|
|
||
| ## Sessions stay alive after work is done | ||
|
|
||
| In the sample, `/ping` reports `HealthyBusy` while an async task is registered, and AgentCore keeps the session alive while it does. If the handler never calls `complete_async_task` when the Worker exits, the session stays `HealthyBusy` until `maxLifetime`. The idle session timeout does not help either, because it resets only on AgentCore Runtime invocations, not on Temporal polling. The fix is a Worker idle policy that drains the Worker and releases the async task in all cases, including errors. The language-specific code is in the selected `sdk-<language>.md`. | ||
|
|
||
| ## Lifecycle configuration rejected | ||
|
|
||
| `ValidationException` on a Runtime create or update means one of these: a lifecycle value is below 60 seconds or above 28800 seconds (microVMs), or `idleRuntimeSessionTimeout` is greater than `maxLifetime`. | ||
|
|
||
| ## Rules | ||
|
|
||
| - Inspect the specific Runtime, endpoint, stack and version named in the inventory. Do not enumerate AgentCore resources across Regions to reconstruct state. | ||
| - Do not change IAM, redeploy, or repoint an endpoint until the evidence above identifies the cause. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you remove the explanation of how this guide was authored (“uses only the gotchas documented in the sources above”) and state the durable operating rule directly? For example: “Use the WCI inspection flow below. For symptoms not covered here, gather the listed evidence and escalate without speculating about the cause.”