Skip to content

fix(aws-bedrockagent): close the real-cloud parity gaps in #1452 - #1559

Merged
thzgajendra merged 5 commits into
stackshy:developmentfrom
thzgajendra:fix/bedrockagent-parity-1452
Oct 15, 2026
Merged

thzgajendra merged 5 commits into
stackshy:developmentfrom
thzgajendra:fix/bedrockagent-parity-1452

Conversation

@thzgajendra

@thzgajendra thzgajendra commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Closes every open item of AWS BedrockAgent: open real-cloud parity gaps #1452: cloudemu now serves all 75 Bedrock Agent (bedrock-agent) operations of the pinned SDK (aws-sdk-go-v2/service/bedrockagent v1.58.0), up from 32, from both the Go library and cloudemu serve.
  • Deploying an agent works the way AWS documents it. An alias created or updated without routingConfiguration cuts the next numbered version of DRAFT, with its action groups, knowledge bases and collaborators. Every agent has TSTALIASID.
  • Items feat(bedrockagent): tags, required-field validation, List paging, 10-char IDs (E1) #1360 already fixed (tags, paging, VECTOR validation, 10-character ids) get regression tests. One item is not a bug, explained under Changes.

Changes

The problem (in plain words)

A team that tests its Bedrock agent setup against cloudemu could create an agent but not deploy it. aws bedrock-agent get-agent-alias, create-agent-action-group, associate-agent-knowledge-base, get-ingestion-job, create-flow-version, create-prompt-version and 37 more operations answered 404 unsupported path. Terraform could not apply aws_bedrockagent_agent_action_group, aws_bedrockagent_agent_knowledge_base_association, aws_bedrockagent_agent_collaborator or aws_bedrockagent_agent_alias (whose read needs GetAgentAlias). Versions did not exist at all, so a pipeline that deploys version N and rolls back to N-1 could not be tested.

How this fixes it

Each missing area is a small optional interface in services/bedrockagent/driver (the pattern #1528 used for ECS), so the main BedrockAgent interface keeps its signature. The AWS mock implements all of them, the wire handler finds them by type assertion, and the portable wrapper exposes them through the same recorder/metrics/rate-limit/error-injection pipeline. Feature files share one name across the three layers:

Feature (<name>.go in driver, provider, server) Operations
agent_versions Get/List/DeleteAgentVersion, Get/List/Update/DeleteAgentAlias (CreateAgentAlias now cuts versions)
action_groups Create/Get/Update/Delete/ListAgentActionGroup(s)
agent_knowledge_bases Associate/Get/Update/Disassociate/ListAgentKnowledgeBase(s)
agent_collaborators Associate/Get/Update/Disassociate/ListAgentCollaborator(s)
ingestion_jobs Get/List/StopIngestionJob (filters, sortBy)
flow_versions Create/Get/List/DeleteFlowVersion, Create/Get/List/Update/DeleteFlowAlias
prompt_versions CreatePromptVersion; promptVersion on GetPrompt/DeletePrompt; promptIdentifier on ListPrompts
documents Ingest/Get/List/DeleteKnowledgeBaseDocuments
flow_validation ValidateFlowDefinition
resource_policies Put/Get/DeleteResourcePolicy (not in #1452; added so all 75 SDK operations are served)

Rules, all in the provider so the library and serve behave the same:

  • Versions are numbered from 1 per agent, flow or prompt, never reused, and are immutable snapshots. Deleting a version that an alias routes to is a ConflictException (409) unless skipResourceInUseCheck is set (DeleteAgentVersion). PrepareAgent cuts no version, as in AWS (Deploy an agent: versions come from aliases), which answers the "PrepareAgent cuts no version" item. Cutting an agent version also prepares DRAFT, which Terraform's alias waiter expects. A version-qualified prompt identifier (prompt/ID:N) deletes only that version, and updating one is a ValidationException.
  • Agent sub-resources are written only on DRAFT (pattern DRAFT, else ValidationException), readable on any version, and a DRAFT change puts the agent back to NOT_PREPARED. Names are unique per agent (ConflictException). Collaborators need a supervisor agent (agentCollaboration SUPERVISOR or SUPERVISOR_ROUTER) and an existing alias. An action group takes apiSchema or functionSchema, not both, and an inline apiSchema.payload must be a JSON or YAML OpenAPI document. DeleteAgent refuses (ConflictException) an agent whose alias another agent's collaborator uses, unless skipResourceInUseCheck is set.
  • Validation copies the service model: name pattern ([0-9a-zA-Z][_-]?){1,100}, description 1..200, clientToken 33..256, enums verbatim, 1..10 documents, alias routing 0..1 (agent) or exactly 1 (flow), flow concurrency 1..100.
  • Errors: FailedPrecondition now maps to ConflictException 409 (it was mapped to ValidationException 400 but no code path used it before) and ResourceExhausted to ServiceQuotaExceededException with the model's 402. Success codes follow the model too: 202 for CreateAgent/aliases/versions/ingestion/documents, 201 for CreateFlow/Prompt and their versions/aliases, 204 for the three deletes.
  • Agent configuration (promptOverrideConfiguration, agentCollaboration, orchestrationType, guardrailConfiguration, memoryConfiguration, customOrchestration, customerEncryptionKeyArn) is stored and returned; an update that omits a block keeps it. This closes the "promptOverrideConfiguration dropped" item; AWS's default prompt templates are not reproduced (non-goals).
  • Identifiers may be ARNs where the model allows it (flowIdentifier, aliasIdentifier, promptIdentifier with :N); the handler splits the escaped path, so an ARN stays one segment.
  • Tags now resolve flow-alias (flow/F/alias/A) and prompt-version (prompt/P:N) ARNs.
  • Ingestion jobs finish at once, or report IN_PROGRESS for two seconds under --async-settle (new settle.DefaultBedrockIngestionSettle); only a running job can be stopped. A data source runs one job at a time: a start while one is running is a ConflictException.
  • Concurrency: one Mock-level mutex covers every compound write: name check + insert, version cut, cascading delete, and the updates and child creates a delete can race. So a PrepareFlow racing DeleteFlow can't bring the flow back. Reads are deep copies; all new stores are in snapshot.go, and agents persisted by older builds read back with the defaults.
  • Routing: /resourcepolicy/{arn} is shared with VPC Lattice, so bedrock-agent claims it only for knowledge-base ARNs (added to the prefix-collision guard test).

Not a bug: "CreateAgent without agentResourceRoleArn accepted" - the service model requires only agentName and the CreateAgent reference marks the role Required: No, so the emulator keeps accepting it (pinned by a test). The exact AWS wording for a missing VECTOR storageConfiguration and a bad nextToken can't be confirmed without a real account; the exception and status are right and the wording is listed in the non-goals.

Deliberate differences from AWS are listed in docs/coverage/nongoals/bedrockagent.md. In short: synchronous states, structural-only flow validation, documents tracked but not embedded, DeleteFlow/DeletePrompt cascade without an in-use check (DeleteAgent checks collaborators that use its aliases), deleting a knowledge base leaves agent associations (and deleting an agent leaves other agents' collaborators pointing at its alias), clientToken is validated but not idempotent, no TSTALIASID for flows.

Diagram

flowchart LR
    CLI["aws CLI / SDK / Terraform"] --> H["server/aws/bedrockagent<br/>restJson1 routes"]
    H -->|"type assertion<br/>(capabilityOf)"| C{"optional capability<br/>AgentVersions, ActionGroups,<br/>FlowVersions, ..."}
    C --> P["providers/aws/bedrockagent Mock<br/>validation, state, versions"]
    P --> S[("memstore stores<br/>+ snapshot.go")]
    W["services/bedrockagent<br/>portable wrapper"] --> C
    subgraph Deploy
      D["DRAFT + action groups + KBs"] -->|"CreateAgentAlias<br/>no routing"| V1["version 1 snapshot"]
      V1 --> A["alias prod -> 1"]
      D -->|"UpdateAgentAlias<br/>no routing"| V2["version 2"] --> A2["alias prod -> 2<br/>history: 1 closed"]
    end
Loading

Provider Coverage

  • AWS

Azure, GCP and OCI removed: Bedrock Agent exists only on AWS.

Checklist

  • All tests pass (go test ./...)
  • Linter passes (golangci-lint run --timeout=9m ./...)
  • Every provider the change applies to implements the same behavior
  • Unit tests added to provider test files
  • Regenerated docs (go generate ./... for coverage; go run ./internal/compatgen for the compat matrix) and committed the result

"Integration tests added to cloudemu_test.go" removed: like recent single-service PRs, the integration tests are real-SDK round trips beside the handler (server/aws/bedrockagent/*_test.go).

Test Plan

How I tested it

Real tools against the built server (go build -race ./cmd/cloudemu, serve --providers aws), with the same scripts also run against a development (8c0bf2d) binary as the baseline:

  • AWS CLI v2 2.30.0, every one of the 72 operations it has, plus raw HTTP for the 3 newer ones and for status codes and X-Amzn-Errortype: 109 checks pass here; 86 of them fail on development. Happy paths, each error with its exception and status (ValidationException 400, ResourceNotFoundException 404, ConflictException 409), paging, cascades, tags on flow-alias and prompt-version ARNs. 0 DATA RACE in the server log.
  • Concurrency over the wire, 50 parallel requests: same-name alias and action-group creates give exactly 1 success and 49 ConflictException; 50 alias creates cut 50 distinct versions; 25 UpdateAgent racing 25 alias creates leave every version a consistent snapshot.
  • boto3 1.43.112 in a fresh venv, a CI-style flow: build an agent with a RETURN_CONTROL action group, a knowledge base and CUSTOM documents; prepare; deploy an alias; bedrock-agent-runtime invoke_agent through the new alias and through TSTALIASID; share the knowledge base with a resource policy (stale revision -> 409); redeploy (version 2 keeps the new model, version 1 the old); the boto3 paginator; teardown. 17/17.
  • Terraform (OpenTofu, hashicorp/aws 6.x, the repo's contrib/terraform/cloudemu-tf): 10 resources (two agents, action group, knowledge base, data source, KB association, alias, collaborator, prompt, flow) apply, plan shows No changes, an in-place update of three resources applies (the alias update cuts version 2, as in AWS), plan shows No changes again, destroy removes all 10.
  • --persist restart: aliases, version snapshots, tags, a stopped job, prompt versions and the version counter survive. --async-settle: a job is IN_PROGRESS, can be stopped, settles to COMPLETE, and a finished job can't be stopped (409). --enforce-auth: a seeded admin can call the new operations; an IAM user with only s3:* gets AccessDeniedException; unsigned requests get 403. Upgrade: state written by the development binary loads here (legacy agents report DISABLED/DEFAULT). Neighbours: VPC Lattice resource policies, an S3 bucket named resourcepolicy, and EKS tags still work on the same server.
  • go install ./cmd/cloudemu into an empty GOBIN: cloudemu start, eval "$(cloudemu env ...)", create agent + alias with the CLI, logs, stop.
  • Not run: the Docker image, because the Docker daemon on my machine was unresponsive.

New tests: real aws-sdk-go-v2 round trips per feature (TestSDKAgentAliasCutsVersions, TestSDKAgentTestAlias, TestSDKActionGroupLifecycle, TestSDKAgentKnowledgeBaseAssociation, TestSDKAgentCollaborators, TestSDKIngestionJobsSync, TestSDKIngestionJobStopWhileRunning, TestSDKFlowVersionsAndAliases, TestSDKDeleteFlowCascades, TestSDKPromptVersions, TestSDKKnowledgeBaseDocuments, TestSDKValidateFlowDefinition, TestSDKResourcePolicy, TestSDKAgentConfigurationRoundTrip) asserting typed exceptions and HTTP status; provider tests for validation, -race concurrency (TestConcurrentAliasCreatesCutDistinctVersions, TestConcurrentSameNameAliasOneWins, TestConcurrentSameNameActionGroupOneWins, TestConcurrentAssociateSameKnowledgeBaseOneWins), deep copies, cascades and persistence (TestSnapshotRoundTripVersionsAndChildren, TestRestoreLegacyAgentReportsDefaults), delete-vs-write races and malformed ids (TestReviewFixes); TestMatchesResourcePolicyOnlyForKnowledgeBases; wrapper tests including error injection. Two existing tests changed for model fidelity: TestAgentAliasRequiresAgent now uses a well-formed missing id ("nope" fails the id pattern first, as in AWS) and TestSharedTagsPathRoutedByARN expects CreateAgent's 202.

Gate: go build -mod=readonly, go vet, go test ./..., go test -race on the touched packages, go mod tidy with no diff, go generate ./... and compatgen with no drift, golangci-lint with 0 new issues on the touched packages (0 before, 0 after), contrib modules build and vet, CodeQL go-security-extended run locally over the whole tree: 0 new findings (the same 11 as on development, all in files this PR does not touch).

Coverage

  • providers/aws/bedrockagent 83.5% -> 90.1%
  • server/aws/bedrockagent 70.6% -> 93.1%
  • services/bedrockagent 36.9% -> 99.8%

Related Issues

Closes #1452

@NitinKumar004 NitinKumar004 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this, it's a big step for Bedrock Agent. I checked out 638ff0f, built cloudemu serve, and ran the AWS CLI and the real hashicorp/aws provider (6.68.0) against it, including a --persist restart. Most of it holds up well: versions are cut and numbered correctly, in-use checks on version deletes work, child resources are copied into versions, and state survives a restart, version counters included. Two things need fixing before merge. One is a Terraform failure on the headline alias flow. The other loses data when a prompt version is deleted by ARN.

Issue coverage (#1452)

Item Status Evidence
Action groups CRUD Fixed CLI create/dup (409)/list on DRAFT and version 1; TF aws_bedrockagent_agent_action_group apply/update/destroy clean
Agent KB associations Fixed CLI associate / unknown KB 404 / duplicate 409; TF association clean
Tags on CreateAgent/KB, Tag/Untag/List Fixed (regression tests) TF tags on the agent round-trips with no drift
Alias lifecycle Get/List/Update/Delete Fixed, with a Terraform gap (High 1) CLI full lifecycle; TSTALIASID listed, can't be updated or deleted
Get/List/StopIngestionJob Fixed CLI get, list with STATUS filter and sortBy, stop of finished job is 409
Flow versions and aliases Fixed CLI create-flow-version, create-flow-alias, delete of in-use version is 409; TF aws_bedrockagent_flow clean
CreatePromptVersion / per-version delete Fixed, with a data-loss bug (High 2) --prompt-version works, versioned ARN deletes the whole prompt
Agent versions, PrepareAgent cuts no version Fixed Versions cut by alias create/update; list/get/delete; 409 when an alias routes to it
CreateAgent without role Not a bug, agreed API reference marks agentResourceRoleArn Required: No
VECTOR KB without storageConfiguration Fixed ValidationException
maxResults/nextToken Fixed (regression tests) Covered by the pagination tests
DeletePrompt response version Fixed {"id":..,"version":"1"}
Tags on flow-alias / prompt-version ARNs Fixed tag-resource on prompt/P:2 round-trips, and survives restart
promptOverrideConfiguration Partially fixed Caller value kept. Default templates not returned, listed as a non-goal
Exact wording for those two errors Not verifiable Exception and status are right, as the non-goals note says
KB documents API Fixed Ingest checks that the document type matches the data source type
Collaborators, ValidateFlowDefinition Fixed Collaborator needs a SUPERVISOR agent (else ValidationException); validate reports a missing Output node and an unknown target
10-char ids Fixed All minted ids are [0-9A-Z]{10}

Findings

High

1. aws_bedrockagent_agent_alias fails when the agent's DRAFT isn't prepared (providers/aws/bedrockagent/agent_versions.go:306-313, cutAgentVersion)
After CreateAgentAlias the provider runs waitAgentVersioned, which polls GetAgent and expects PREPARED, with VERSIONING as the pending state. cutAgentVersion marks the new version PREPARED but leaves the DRAFT agent as it was. So if the agent is NOT_PREPARED at that point, apply fails at once with waiting for creation Amazon Bedrock Agents Agent Version (...): unexpected state 'NOT_PREPARED', wanted target 'PREPARED'. That happens with prepare_agent = false on the agent, and also when the last child resource changed the DRAFT without preparing it. The resource gets tainted. Every retry cuts another version (my alias ended up on version 5), and the apply only goes through after a manual aws bedrock-agent prepare-agent. CLI repro: create-agent, then create-agent-alias with no prepare step, and get-agent still reports NOT_PREPARED. Fix: when a version is cut from DRAFT, prepare DRAFT in the same locked section (status PREPARED, preparedAt set). Optionally report VERSIONING under --async-settle. Then add a TF-shaped test: unprepared agent, create alias, GetAgent returns PREPARED.

2. DeletePrompt with a version-qualified ARN deletes the whole prompt (providers/aws/bedrockagent/prompts.go:117)
The DeletePrompt identifier pattern allows ...:prompt/ID:N (API_agent_DeletePrompt). GetPrompt treats that suffix as a version. DeletePrompt throws the suffix away and deletes DRAFT and every version. Repro: delete-prompt --prompt-identifier arn:aws:bedrock:us-east-1:000000000000:prompt/O6HW141D6T:2 returned {"id":"O6HW141D6T"}, and afterwards get-prompt on the prompt is 404. Someone cleaning up one old version loses the whole prompt. Fix: when splitPromptIdentifier returns a version, delete just that version, as promptVersion does. Also return 400 if the suffix and promptVersion disagree.

Medium

3. UpdatePrompt with a version-qualified ARN quietly updates DRAFT (providers/aws/bedrockagent/prompts.go:83)
Same root cause. Versions are immutable, and a caller who passes prompt/ID:N expects either that version or an error. It should be a ValidationException, not a write to DRAFT.

4. Deleting an agent whose alias a supervisor uses leaves a collaborator pointing at nothing, and PrepareAgent on the supervisor still succeeds (providers/aws/bedrockagent/agents.go:146)
The non-goals mention this, but it's the main in-use case DeleteAgent's skipResourceInUseCheck exists for (API_agent_DeleteAgent). Today the parameter is ignored. Repro: associate a collaborator via alias LBO6XG0XGE/O4G630FCJG, delete agent LBO6XG0XGE, and the supervisor still lists the collaborator and prepares fine. A multi-agent test suite can't catch a teardown done in the wrong order. Suggest: ConflictException unless skipResourceInUseCheck=true, using the same aliasRoutingTo-style lookup over m.collaborators.

Low

  1. CreateAgentActionGroup takes apiSchema and functionSchema together, any apiSchema.payload text ("abc" was accepted), and a function schema with no executor (providers/aws/bedrockagent/action_groups.go:226). The API reference treats these as alternatives. At minimum, reject both schemas at once.
  2. A non-test agent alias can route to DRAFT (update-agent-alias --routing-configuration agentVersion=DRAFT succeeded). In Bedrock only TSTALIASID routes to DRAFT. Worth confirming and rejecting.
  3. StartIngestionJob starts a second job while one is still IN_PROGRESS under --async-settle (providers/aws/bedrockagent/ingestion_jobs.go:31). Bedrock allows one running job per data source. An S3 data source over a bucket that doesn't exist also ends COMPLETE with zero statistics.
  4. A PINECONE storageConfiguration with no pineconeConfiguration is accepted. This predates the PR, so a follow-up is fine.
  5. golangci-lint 2.14.0 flags goprintffuncname at providers/aws/bedrockagent/flow_validation.go:87 (add should be addf). CI's Lint job passed, so this is probably a version difference, but renaming it is a one-line change.
  6. The AWS API reference now lists 79 operations, adding Create/Get/List/DeleteVpcConfiguration. SDK v1.58.0, which the repo pins, has 75, and the PR serves all of those. The "all operations" claim is true for the pinned SDK only. Say so in the coverage notes.
  7. Coverage: server/aws/bedrockagent is at 76.2% and services/bedrockagent at 75.0%, under the 90% target. The provider is at 92.2%.

Terraform (hashicorp/aws 6.68.0, endpoint override bedrockagent)

Resources: agent (x2), agent_alias (x2), agent_action_group (function schema, RETURN_CONTROL), knowledge_base (OpenSearch Serverless), data_source (S3), agent_knowledge_base_association, agent_collaborator, prompt, flow.

  • First apply: 10 of 11 created. The supervisor's alias failed as described in High 1, with no hang.
  • After a manual prepare-agent: apply OK, plan -detailed-exitcode = 0.
  • Updated description/tags/collaboration instruction on all 10 updatable resources: apply OK (10 changed), plan = 0.
  • Destroy: 11 destroyed. Afterwards GetAgent, GetKnowledgeBase, GetPrompt, GetFlow, GetAgentVersion and ListTagsForResource all return ResourceNotFoundException.

CLI (aws-cli 2.31.19)

I ran every family plus its error cases: duplicate names (409), writes to a non-DRAFT version, unknown KB/version/job (404), deleting an in-use agent or flow version (409), updating or deleting TSTALIASID (400), stopping a finished job (409), a non-supervisor collaborator (400), a VECTOR KB without storage (400), and a document type mismatch (400). With --persist, I restarted the server and checked that agent versions, alias routing and history, version-scoped action groups, KB associations, collaborators, ingestion jobs, flow versions and aliases, prompt versions and version-ARN tags all came back. The next cut after the restart was version 3 for both the agent and the prompt, so numbers aren't reused.

What I verified

  • go vet and gofmt are clean on the touched packages. go test -race passes for the provider, server, service and settle packages. TestEveryHandlerDeclaresAuthorization, the prefix-collision and tags-dispatch tests in server/aws, and persist/ all pass. Authorization stays at service level (bedrock:*), the same as before this PR.
  • Running go run ./internal/coveragegen again leaves no diff. All CI checks are green.
  • I ran a concurrency probe with 20 parallel rounds of action-group creates, alias creates (which cut versions), prepares and lists, under -race with -count=3. No races and no duplicate version numbers.
  • I sampled the provider closely: agent versions, aliases, ingestion jobs, prompt/flow deletes, action-group validation, and DeleteAgent cascade. I read documents, resource policies and flow validation only lightly.

Verdict: request changes for High 1 and High 2, plus the Medium items.


snap := cloneAgent(draft)
snap.Version = version
snap.Status = agentVersionPrepared

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This marks the new version PREPARED but leaves DRAFT as it was. Terraform's aws_bedrockagent_agent_alias runs waitAgentVersioned after CreateAgentAlias and wants GetAgent to be PREPARED, so an alias on a NOT_PREPARED agent fails at once with unexpected state 'NOT_PREPARED', wanted target 'PREPARED'. Every retry cuts another version. Please prepare DRAFT here too (status PREPARED and preparedAt), since cutting a version is when Bedrock prepares it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in deee77b. Cutting a version now prepares DRAFT in the same locked section (status PREPARED, preparedAt set), so waitAgentVersioned sees PREPARED straight away. TestAliasCutPreparesDraft covers it: an unprepared agent, an alias, then GetAgent is PREPARED and only version 1 exists. I also re-ran your Terraform case: an agent and action group with prepare_agent = false, then an alias, on hashicorp/aws 6.69.0. On 638ff0f it fails with unexpected state 'NOT_PREPARED', wanted target 'PREPARED'. With the fix it applies 4 resources, cuts one version, plan -detailed-exitcode is 0, and destroy is clean.

Comment thread providers/aws/bedrockagent/prompts.go Outdated
// DeletePrompt deletes a prompt, every version of it and their tags, and
// returns its identifier.
func (m *Mock) DeletePrompt(_ context.Context, id string) (string, error) {
id, _ = splitPromptIdentifier(id)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The :N suffix is dropped here, so delete-prompt --prompt-identifier arn:...:prompt/ID:2 deletes DRAFT and every version instead of version 2 (I confirmed this with the CLI). The DeletePrompt pattern allows the suffix and GetPrompt already treats it as a version. Please delete just that version when a version is present.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. A version-qualified identifier now deletes only that version, and the response is {"id", "version"}. If promptVersion disagrees with the suffix, the request is a ValidationException. Tests: TestSDKVersionQualifiedPromptIdentifier (SDK) and TestVersionQualifiedPromptIdentifierInTheLibrary (Go library). Both fail on the old code. With the CLI, delete-prompt --prompt-identifier <arn>:2 now leaves the prompt, with DRAFT and version 1 still listed.

Comment thread providers/aws/bedrockagent/prompts.go Outdated
//
//nolint:gocritic // cfg matches the driver interface signature; copied once on entry.
func (m *Mock) UpdatePrompt(_ context.Context, id string, cfg driver.PromptConfig) (*driver.Prompt, error) {
id, _ = splitPromptIdentifier(id)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing for UpdatePrompt: prompt/ID:N ends up updating DRAFT. Versions are immutable, so a version-qualified identifier should be a ValidationException here.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed. UpdatePrompt with a version-qualified identifier is a ValidationException: versions are immutable, so the message points to DRAFT instead.

// share.
//
//nolint:gocritic // cfg matches the driver interface signature.
func validateActionGroup(cfg driver.ActionGroupConfig) error {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing here checks schema and executor combinations: apiSchema and functionSchema together, any apiSchema payload text, and a function schema with no executor are all accepted. Rejecting both schemas at once would at least catch the common config mistake.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the part you asked for. Create and UpdateAgentActionGroup now reject apiSchema with functionSchema, and reject an inline apiSchema.payload that is not an OpenAPI document in JSON or YAML ("abc" is rejected). A YAML payload is accepted. A function schema with no executor is still accepted, because the model doesn't require an executor; I'd rather not reject it without an AWS source.

found []driver.FlowValidation
}

func (c *flowChecks) add(typ, format string, args ...any) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golangci-lint 2.14.0 flags goprintffuncname here. Renaming to addf fixes it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed to addf.

…letes and schemas

- Cutting an agent version (an alias created or updated without
  routingConfiguration) prepares the DRAFT too, so Terraform's
  aws_bedrockagent_agent_alias waiter sees PREPARED instead of failing on
  an unprepared agent and cutting a new version on every retry.
- DeletePrompt with a version-qualified identifier (prompt/ID:N) deletes
  only that version; a promptVersion that disagrees with the suffix, and
  UpdatePrompt on a version, are ValidationExceptions.
- DeleteAgent honours skipResourceInUseCheck: an agent whose alias another
  agent's collaborator uses is a ConflictException unless it is set
  (new optional AgentInUseDeleter capability; DeleteAgent is unchanged).
- CreateAgentActionGroup rejects apiSchema and functionSchema together and
  an apiSchema payload that is not a JSON or YAML document.
- StartIngestionJob is a ConflictException while a job of the same data
  source is still running.
- Rename flowChecks.add to addf (goprintffuncname).
- Tests for every item; server/aws/bedrockagent coverage 78.7% -> 93.1%,
  services/bedrockagent 74.9% -> 99.8%. Docs: non-goals and services.md.
@thzgajendra

Copy link
Copy Markdown
Collaborator Author

Thanks for the thorough review and the Terraform run. Everything below is in deee77b, with a docs follow-up in 02e4bf8. The branch also merged development 39b938f, which is the merge commit before it.

Fixed

  • High 1: cutting an agent version prepares DRAFT, so the Terraform alias flow works on an unprepared agent. I reproduced the failure on the old code with hashicorp/aws 6.69.0. The fix applies cleanly, with no drift and one version cut.
  • High 2 and Medium 3: prompt/ID:N deletes only that version, a mismatch with promptVersion is 400, and an update of a version is 400.
  • Medium 4: DeleteAgent honours skipResourceInUseCheck. An agent whose alias another agent's collaborator uses is a ConflictException unless the flag is set. This goes through a new optional AgentInUseDeleter capability, so DeleteAgent(ctx, id) keeps its signature for library users. A supervisor deleting itself is not "in use".
  • Low 5: both schemas at once, or a payload that isn't a JSON/YAML document, is a ValidationException.
  • Low 7 (first half): StartIngestionJob is a ConflictException while a job of the same data source is still running. I checked this live with --async-settle. The second half, a missing bucket ending COMPLETE, is fixed in fix(aws-bedrockagentruntime): close the real-cloud parity gaps in #1453 #1561: ingestion jobs there read S3, so a bucket that can't be listed marks the job FAILED with failureReasons.
  • Low 9: addf.
  • Low 10: the non-goals now say the 75 operations are those of the pinned SDK v1.58.0, and that the newer VpcConfiguration operations are not served.
  • Low 11: coverage of server/aws/bedrockagent went from 78.7% to 93.1%, and services/bedrockagent from 74.9% to 99.8%. The new tests cover every family with unknown ids, the update round trips, wrong methods and broken bodies on every route, and the wrapper's recorder, metrics, latency, error injection and rate limit.

Not changed, with the reason

  • Low 6 (an alias routing to DRAFT): the routing pattern in the API model allows DRAFT, and I found no AWS source saying it's rejected, so I left it and noted it in the non-goals. Happy to change it if you have a reference.
  • Low 8 (PINECONE without pineconeConfiguration): this predates the PR, as you said. I'll put it in a follow-up issue.

Verification

  • Every fix has a test shown to fail without it.
  • go test -race passes on the touched packages, and golangci-lint on new lines reports 0 issues.
  • The AWS CLI suite (109 rows) and the boto3 journey (17 rows) pass, plus 14 new CLI rows for these fixes and the Terraform run above.
  • CodeQL reports nothing in the changed files; its 11 findings are in four files this PR doesn't touch.
  • CI is green on deee77b.

@thzgajendra

Copy link
Copy Markdown
Collaborator Author

Follow-up for Low 8 (a storage type without its configuration block): #1642.

…to fix/bedrockagent-parity-1452

# Conflicts:
#	docs/coverage/aws/bedrockagent.md
#	docs/coverage/nongoals/bedrockagent.md
#	docs/services.md
#	internal/settle/settle.go
#	providers/aws/bedrockagent/bedrockagent.go
#	providers/aws/bedrockagent/documents.go
#	providers/aws/bedrockagent/documents_test.go
#	providers/aws/bedrockagent/ingestion_jobs.go
#	providers/aws/bedrockagent/ingestion_jobs_test.go
#	server/aws/bedrockagent/knowledge_bases.go
#	server/aws/bedrockagent/types.go
#	server/aws/restjson_prefix_collision_test.go
#	services/bedrockagent/driver/documents.go
#	services/bedrockagent/driver/driver.go

@NitinKumar004 NitinKumar004 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick turnaround. This branch went into development with #1561, so I reviewed development at c7cd389 instead of the PR head. I built cloudemu serve from it and reran the round-1 cases with the AWS CLI. I also ran the full Terraform flow (hashicorp/aws 6.68.0) without a manual prepare. Both Highs and both Mediums are fixed. What's left are a few Lows, two of them new and small.

Round-1 items

# Item Status Evidence
H1 Alias on an unprepared agent fails in Terraform Fixed CLI: create-agent (NOT_PREPARED), then create-agent-alias, and get-agent reports PREPARED with preparedAt set. After an action-group change put DRAFT back to NOT_PREPARED, update-agent-alias cut version 2 and prepared DRAFT again. Terraform: the prepare_agent = false agent plus its alias applied on the first run, with exactly one version cut (DRAFT 1)
H2 DeletePrompt with prompt/ID:N deletes the whole prompt Fixed delete-prompt arn:...:prompt/P:2 removed only version 2. DRAFT and version 1 stayed. A promptVersion that disagrees with the suffix is a ValidationException, and an unknown suffix version is a 404. See Low 1 for the response body
M3 UpdatePrompt with prompt/ID:N writes to DRAFT Fixed ValidationException for both the ARN and the bare ID:N forms. Plain UpdatePrompt still returns version: DRAFT
M4 DeleteAgent ignores skipResourceInUseCheck Fixed ConflictException (409) by default and with --no-skip-resource-in-use-check, including when only the supervisor's version 1 references the alias. --skip-resource-in-use-check deletes it (202). A supervisor can delete itself. This matches the default-false wording in DeleteAgent
L5 Action-group schema checks Partly fixed apiSchema plus functionSchema together is rejected on both create and update. A payload of abc or [1,2] is rejected, and a real YAML OpenAPI document is accepted. A functionSchema with no executor is still accepted. That's fine since actionGroupExecutor is optional in the API model
L6 Regular alias routing to DRAFT Not changed, agreed The routing agentVersion pattern allows DRAFT, and it's written up in the non-goals
L7 Second ingestion job while one runs, missing bucket ends COMPLETE Fixed Under --async-settle, a second start on the same data source is a 409 while the first is IN_PROGRESS. Another data source can still start in parallel. A start after a stop works. A data source on a missing bucket ends FAILED with failureReasons
L8 PINECONE without its config block Deferred Tracked in #1642
L9 goprintffuncname Fixed addf. There's a new lint hit, see Low 2
L10 Newer VpcConfiguration ops Documented The non-goals now say 75 ops of the pinned SDK v1.58.0
L11 Coverage Fixed provider 92.3%, server 91.8%, services 99.8%
- promptOverrideConfiguration default templates Documented Non-goals line 4

New findings

No High or Medium.

Low

  1. A version delete by qualified ARN echoes ID:N as the prompt id (server/aws/bedrockagent/prompt_versions.go:56). delete-prompt --prompt-identifier arn:...:prompt/P8KRSZQH27:2 returns {"id": "P8KRSZQH27:2", "version": "2"}. DeletePrompt defines id as [0-9a-zA-Z]{10}. resourceIDOf strips the ARN prefix but keeps the :N. A caller that reads id back, such as for a later GetPrompt or a tag lookup, gets an id that doesn't match. Fix: use the id from splitPromptIdentifier here. The library path already does this.
  2. servePromptItem is now at gocyclo 11 (server/aws/bedrockagent/prompts.go:36). golangci-lint 2.14.0 on the three bedrockagent packages reports this as the only issue, and the suffix block added in the fix is what pushed it over. Moving the suffix/promptVersion reconciliation into a small helper fixes it.
  3. Minor, already documented: the library's plain DeleteAgent still skips the in-use check, while AWS defaults to checking. That keeps the signature stable, and the non-goals call it out, so I'm only noting it.

Terraform (hashicorp/aws 6.68.0, endpoints bedrockagent, s3)

Resources: 3 agents (one with prepare_agent = false, one SUPERVISOR also with prepare_agent = false), 3 aliases, an action group (function schema, RETURN_CONTROL), an S3 bucket, a knowledge base (OpenSearch Serverless), an S3 data source, a KB association, a collaborator, a prompt and a flow. That's 14 resources.

  • Apply: 14 added on the first run. No prepare-agent call, no waiter error, no retries. Every alias is on version 1.
  • plan -detailed-exitcode: 0.
  • Changed description, tags and collaboration instruction: 13 changed, then plan 0.
  • Destroy: 14 destroyed. The collaborator came off before its agent, so the new in-use check didn't block the teardown. Afterwards GetAgent on both agents, GetAgentVersion, GetKnowledgeBase, GetPrompt and GetFlow all return ResourceNotFoundException.

CLI (aws-cli 2.31.19)

Prompt version delete and update in ARN, bare and mismatched forms, plus the DRAFT suffix (400 from the [0-9]{1,5} pattern). DeleteAgent with the flag set, unset and false, against a DRAFT collaborator and a version-1 collaborator, and a supervisor deleting itself. Action-group schema combinations on create and update. Ingestion concurrency and failure under --async-settle. The alias version-cut and prepare sequence described above.

What I verified

  • go vet and gofmt are clean. go test -race -p 1 passes on providers/aws/bedrockagent, server/aws/bedrockagent and services/bedrockagent. go run ./internal/coveragegen leaves no diff.
  • Race probes under -race -count=3:
    • 30 rounds of DeleteAgent with the check on, racing AssociateAgentCollaborator to the same alias plus PrepareAgent on the supervisor. Delete and associate never both succeeded, so no dangling collaborator.
    • 10 rounds of version-qualified DeletePrompt racing UpdatePrompt on DRAFT, CreatePromptVersion, and UpdatePrompt on a version. Exactly the deleted versions were gone, every other version and the prompt remained, and every version update was rejected. No data races.
  • The fix commits deee77b and 02e4bf8 carry no attribution trailers.
  • I checked the fix delta against the API reference for DeletePrompt, UpdatePrompt and DeleteAgent.

These changes already shipped in development through #1561, so this PR has nothing left to merge. Once the two Lows above land, as a small follow-up or as part of #1642, I think #1559 can be closed.

Verdict: approve. The round-1 Highs and Mediums are all resolved, and the remaining items are Low.

@thzgajendra
thzgajendra merged commit c934949 into stackshy:development Oct 15, 2026
20 checks passed
@thzgajendra
thzgajendra deleted the fix/bedrockagent-parity-1452 branch October 15, 2026 13:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AWS BedrockAgent: open real-cloud parity gaps

2 participants