Skip to content

fix(objectstore): sign GetBucketLocation for AWS global endpoint - #4112

Open
anxkhn wants to merge 1 commit into
kanisterio:masterfrom
anxkhn:loop/kanister__001
Open

anxkhn wants to merge 1 commit into
kanisterio:masterfrom
anxkhn:loop/kanister__001

Conversation

@anxkhn

@anxkhn anxkhn commented Jun 30, 2026

Copy link
Copy Markdown

Change Overview

When a profile sets an explicit endpoint of s3.amazonaws.com together with a
non us-east-1 region (e.g. eu-west-1), bucket region validation fails:

GetBucketLocation: AuthorizationHeaderMalformed: The authorization header is
malformed; the region 'eu-west-1' is wrong; expecting 'us-east-1'

s3.amazonaws.com is the legacy global (us-east-1) S3 endpoint, so requests to
it must be signed for us-east-1. The SDK was signing GetBucketLocation with
the user-supplied region, which S3 rejects. This broke kanctl create profile
and kando whenever an AWS endpoint was provided for a bucket outside
us-east-1; the documented workaround is to drop --endpoint.

This overrides the signing region to us-east-1 for that single
GetBucketLocation call, gated to the AWS global endpoint only. Region-specific
AWS endpoints (s3.<region>.amazonaws.com) and s3-compatible stores
(MinIO/Ceph) keep the configured region, so TestInvalidS3RegionEndpointMismatch
and existing behaviour are unaffected.

Pull request type

Please check the type of change your PR introduces:

  • 🚧 Work in Progress
  • 🌈 Refactoring (no functional changes, no api changes)
  • 🐹 Trivial/Minor
  • 🐛 Bugfix
  • 🌻 Feature
  • 🗺️ Documentation
  • 🤖 Test
  • 🏗️ Build

Issues

Test Plan

  • ⚡ Unit test - added a credential-free TestIsAWSGlobalS3Endpoint
    covering scheme-less / scheme / trailing-slash global endpoints, region-specific
    AWS endpoints, MinIO/Ceph, and empty input.
  • 💚 E2E - the existing live-AWS objectstore suite is exercised in
    CI (it requires AWS creds / MinIO and is skipped without them).
go test ./pkg/objectstore/                # global-endpoint detection runs; live-AWS suites skip
gofmt -l pkg/objectstore/bucket.go        # clean

@anxkhn
anxkhn force-pushed the loop/kanister__001 branch from 445ab56 to 8770581 Compare July 1, 2026 07:40
@anxkhn
anxkhn force-pushed the loop/kanister__001 branch from 8770581 to 267cca8 Compare August 12, 2026 05:24
When a profile sets an explicit endpoint of s3.amazonaws.com together with a
non us-east-1 region (e.g. eu-west-1), bucket region validation failed with:

  GetBucketLocation: AuthorizationHeaderMalformed: ... the region 'eu-west-1'
  is wrong; expecting 'us-east-1'

s3.amazonaws.com is the legacy global (us-east-1) endpoint, so requests to it
must be signed for us-east-1; the SDK was signing GetBucketLocation with the
user-supplied region instead. This broke `kanctl create profile` and `kando`
whenever an AWS endpoint was provided for a bucket outside us-east-1.

Override the signing region to us-east-1 for the single GetBucketLocation call
only when the configured endpoint is the AWS global endpoint. Region-specific
AWS endpoints (s3.<region>.amazonaws.com) and s3-compatible stores (MinIO/Ceph)
keep the configured region, so TestInvalidS3RegionEndpointMismatch and existing
behaviour are unaffected.

Add a credential-free unit test for the global-endpoint detection; the existing
objectstore suite still requires live AWS/MinIO and is exercised in CI.

Ref kanisterio#3613

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn
anxkhn force-pushed the loop/kanister__001 branch from 267cca8 to 4028de6 Compare August 13, 2026 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant