-
Notifications
You must be signed in to change notification settings - Fork 91
add tests for server side encryption, file backend encryption only #2369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| Feature: Server Side Encryption | ||
|
|
||
| @2.14.0 | ||
| @PreMerge | ||
| @ServerSideEncryption | ||
| @ServerSideEncryptionFileBackend | ||
| Scenario Outline: should encrypt object when bucket encryption is <bucketAlgo> and object encryption is <objectAlgo> | ||
| Given a "Non versioned" bucket | ||
| And bucket encryption is set to "<bucketAlgo>" with key "<bucketKeyId>" | ||
| Then the bucket encryption is verified for algorithm "<bucketAlgo>" and key "<bucketKeyId>" | ||
| When an object "<objectName>" is uploaded with SSE algorithm "<objectAlgo>" and key "<objectKeyId>" | ||
| Then the PutObject response should have SSE algorithm "<expectedAlgo>" and KMS key "<expectedKeyId>" | ||
| Then the GetObject should return the uploaded body with SSE algorithm "<expectedAlgo>" and KMS key "<expectedKeyId>" | ||
|
|
||
| Examples: No bucket encryption | ||
| | objectName | bucketAlgo | bucketKeyId | objectAlgo | objectKeyId | expectedAlgo | expectedKeyId | | ||
| | no-enc-none | | | | | | absent | | ||
| | no-enc-aes | | | AES256 | | AES256 | absent | | ||
| | no-enc-kms | | | aws:kms | | aws:kms | generated | | ||
| | no-enc-kms-key | | | aws:kms | custom-key-1 | aws:kms | custom-key-1 | | ||
|
|
||
| Examples: Bucket AES256 | ||
| | objectName | bucketAlgo | bucketKeyId | objectAlgo | objectKeyId | expectedAlgo | expectedKeyId | | ||
| | bkt-aes-none | AES256 | | | | AES256 | absent | | ||
| | bkt-aes-aes | AES256 | | AES256 | | AES256 | absent | | ||
| | bkt-aes-kms | AES256 | | aws:kms | | aws:kms | generated | | ||
| | bkt-aes-kms-key | AES256 | | aws:kms | custom-key-1 | aws:kms | custom-key-1 | | ||
|
|
||
| Examples: Bucket aws:kms (default key) | ||
| | objectName | bucketAlgo | bucketKeyId | objectAlgo | objectKeyId | expectedAlgo | expectedKeyId | | ||
| | bkt-kms-none | aws:kms | | | | aws:kms | generated | | ||
| | bkt-kms-aes | aws:kms | | AES256 | | AES256 | absent | | ||
| | bkt-kms-kms | aws:kms | | aws:kms | | aws:kms | generated | | ||
| | bkt-kms-kms-key | aws:kms | | aws:kms | custom-key-1 | aws:kms | custom-key-1 | | ||
|
|
||
| Examples: Bucket aws:kms with custom key | ||
| | objectName | bucketAlgo | bucketKeyId | objectAlgo | objectKeyId | expectedAlgo | expectedKeyId | | ||
| | bkt-kmskey-none | aws:kms | bucket-key | | | aws:kms | bucket-key | | ||
| | bkt-kmskey-aes | aws:kms | bucket-key | AES256 | | AES256 | absent | | ||
| | bkt-kmskey-kms | aws:kms | bucket-key | aws:kms | | aws:kms | bucket-key | | ||
| | bkt-kmskey-kms-key | aws:kms | bucket-key | aws:kms | custom-key-2 | aws:kms | custom-key-2 | | ||
|
|
||
| @2.14.0 | ||
| @PreMerge | ||
| @ServerSideEncryption | ||
| @ServerSideEncryptionFileBackend | ||
| Scenario: DeleteBucketEncryption removes default encryption | ||
| Given a "Non versioned" bucket | ||
| And bucket encryption is set to "AES256" with key "" | ||
| When an object "enc-obj" is uploaded with SSE algorithm "" and key "" | ||
| Then the GetObject should return the uploaded body with SSE algorithm "AES256" and KMS key "absent" | ||
| When the user deletes bucket encryption | ||
| Then the GetObject should return the uploaded body with SSE algorithm "AES256" and KMS key "absent" | ||
| When an object "plain-obj" is uploaded with SSE algorithm "" and key "" | ||
| Then the GetObject should return the uploaded body with SSE algorithm "" and KMS key "absent" | ||
|
SylvainSenechal marked this conversation as resolved.
|
||
|
|
||
| @2.14.0 | ||
| @PreMerge | ||
| @ServerSideEncryption | ||
| @ServerSideEncryptionFileBackend | ||
| Scenario Outline: PutObject with invalid SSE parameters returns an error: <objectName> | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These last 3 scenarios are maybe not so relevant for functional tests, test in cloudserver would probably be enough but I guess its fine
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you double check we have the tests in Cloudserver, though?
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. half of them are covered, so yeah it's very moderately useful
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. so half are not covered : can you create a ticket in cloudserver? |
||
| Given a "Non versioned" bucket | ||
| When an object "<objectName>" is uploaded with SSE algorithm "<algo>" and key "<keyId>" | ||
| Then it should fail with error "InvalidArgument" | ||
|
|
||
| Examples: | ||
| | objectName | algo | keyId | | ||
| | sse-invalid-algo | INVALID_ALGO | | | ||
| | sse-aes-kms-err | AES256 | some-key | | ||
|
|
||
| @2.14.0 | ||
| @PreMerge | ||
| @ServerSideEncryption | ||
| @ServerSideEncryptionFileBackend | ||
| Scenario: PutBucketEncryption AES256 with KMS key returns an error | ||
| Given a "Non versioned" bucket | ||
| When bucket encryption is set to "AES256" with key "some-key" | ||
| Then it should fail with error "InvalidArgument" | ||
|
|
||
| @2.14.0 | ||
| @PreMerge | ||
| @ServerSideEncryption | ||
| @ServerSideEncryptionFileBackend | ||
| Scenario: GetBucketEncryption on non-encrypted bucket returns an error | ||
| Given a "Non versioned" bucket | ||
| When the user gets bucket encryption | ||
| Then it should fail with error "ServerSideEncryptionConfigurationNotFoundError" | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,210 @@ | ||
| import { When, Then } from '@cucumber/cucumber'; | ||
| import { CacheHelper, Identity, S3 } from 'cli-testing'; | ||
| import { | ||
| S3Client, | ||
| GetObjectCommand, | ||
| PutObjectCommand, | ||
| type ServerSideEncryption, | ||
| } from '@aws-sdk/client-s3'; | ||
| import Zenko from 'world/Zenko'; | ||
| import assert from 'assert'; | ||
|
|
||
| // We use the AWS SDK directly instead of cli-testing for PutObject and GetObject | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is more annoying than I thought, even later if we wanna parallelize coldStorage test, one of the test is using getObject.. |
||
| // because: | ||
| // - cli-testing has a casing bug: --ssekms-key-id → SsekmsKeyId (should be | ||
| // SSEKMSKeyId), so the KMS key id is silently dropped on PutObject. | ||
| // - cli-testing's S3.getObject writes the body to a shared file (out.loc) | ||
| // which races under parallel execution. | ||
|
francoisferrand marked this conversation as resolved.
|
||
| // Long term solution : consider dropping cli-testing sdk wrapper : https://scality.atlassian.net/browse/ZENKO-5247 | ||
| function buildS3Client(): S3Client { | ||
| const credentials = Identity.getCurrentCredentials(); | ||
| const ssl = CacheHelper.parameters?.ssl === false | ||
| ? 'http://' : 'https://'; | ||
| const host = CacheHelper.parameters?.ip | ||
| || `s3.${CacheHelper.parameters?.subdomain}`; | ||
| const port = CacheHelper.parameters?.port || '80'; | ||
| return new S3Client({ | ||
| region: 'us-east-1', | ||
| endpoint: `${ssl}${host}:${port}`, | ||
| forcePathStyle: true, | ||
| credentials: { | ||
| accessKeyId: credentials.accessKeyId, | ||
| secretAccessKey: credentials.secretAccessKey, | ||
| ...(credentials.sessionToken | ||
| ? { sessionToken: credentials.sessionToken } | ||
| : {}), | ||
| }, | ||
| tls: CacheHelper.parameters?.ssl !== false, | ||
| maxAttempts: 1, | ||
| }); | ||
| } | ||
|
|
||
| When('bucket encryption is set to {string} with key {string}', | ||
| async function (this: Zenko, algo: string, keyId: string) { | ||
| if (!algo) { | ||
| return; | ||
| } | ||
| this.resetCommand(); | ||
| this.addCommandParameter({ bucket: this.getSaved<string>('bucketName') }); | ||
| this.addCommandParameter({ | ||
| serverSideEncryptionConfiguration: JSON.stringify({ | ||
| Rules: [{ | ||
| ApplyServerSideEncryptionByDefault: { | ||
| SSEAlgorithm: algo, | ||
| ...(keyId ? { KMSMasterKeyID: keyId } : {}), | ||
| }, | ||
| }], | ||
| }), | ||
| }); | ||
| const result = await S3.putBucketEncryption(this.getCommandParameters()); | ||
| this.setResult(result); | ||
| }, | ||
| ); | ||
|
|
||
| When('the user gets bucket encryption', async function (this: Zenko) { | ||
| this.resetCommand(); | ||
| this.addCommandParameter({ bucket: this.getSaved<string>('bucketName') }); | ||
| this.setResult(await S3.getBucketEncryption(this.getCommandParameters())); | ||
| }); | ||
|
|
||
| Then('the bucket encryption is verified for algorithm {string} and key {string}', | ||
| async function (this: Zenko, algo: string, keyId: string) { | ||
| if (!algo) { | ||
| return; | ||
| } | ||
| this.resetCommand(); | ||
| this.addCommandParameter({ bucket: this.getSaved<string>('bucketName') }); | ||
| const result = await S3.getBucketEncryption(this.getCommandParameters()); | ||
| assert.ifError(result.err); | ||
| const parsed = JSON.parse(result.stdout) as { | ||
| ServerSideEncryptionConfiguration?: { | ||
| Rules?: Array<{ | ||
| ApplyServerSideEncryptionByDefault?: { | ||
| SSEAlgorithm?: string; | ||
| KMSMasterKeyID?: string; | ||
| }; | ||
| }>; | ||
| }; | ||
| }; | ||
| const defaults = parsed.ServerSideEncryptionConfiguration | ||
| ?.Rules?.[0]?.ApplyServerSideEncryptionByDefault; | ||
| assert.strictEqual(defaults?.SSEAlgorithm, algo, | ||
| `GetBucketEncryption: expected "${algo}", got "${defaults?.SSEAlgorithm}"`); | ||
| if (keyId) { | ||
| assert.ok(defaults?.KMSMasterKeyID, | ||
| 'GetBucketEncryption: KMSMasterKeyID should be present'); | ||
| } else { | ||
| assert.strictEqual(defaults?.KMSMasterKeyID, undefined, | ||
| `GetBucketEncryption: KMSMasterKeyID should be absent, got "${defaults?.KMSMasterKeyID}"`); | ||
| } | ||
| }, | ||
| ); | ||
|
|
||
| When('the user deletes bucket encryption', async function (this: Zenko) { | ||
| this.resetCommand(); | ||
| this.addCommandParameter({ bucket: this.getSaved<string>('bucketName') }); | ||
| const result = await S3.deleteBucketEncryption(this.getCommandParameters()); | ||
| assert.ifError(result.err); | ||
| this.setResult(result); | ||
| }); | ||
|
|
||
| When('an object {string} is uploaded with SSE algorithm {string} and key {string}', | ||
| async function (this: Zenko, objectName: string, algo: string, keyId: string) { | ||
| const SSE_TEST_BODY = 'I am an encrypted test content :-)'; | ||
| this.addToSaved('objectName', objectName); | ||
| this.addToSaved('objectBody', SSE_TEST_BODY); | ||
| const bucket = this.getSaved<string>('bucketName'); | ||
| const client = buildS3Client(); | ||
| try { | ||
| const resp = await client.send(new PutObjectCommand({ | ||
| Bucket: bucket, | ||
| Key: objectName, | ||
| Body: SSE_TEST_BODY, | ||
| ...(algo ? { ServerSideEncryption: algo as ServerSideEncryption } : {}), | ||
| ...(keyId ? { SSEKMSKeyId: keyId } : {}), | ||
| })); | ||
| this.saveCreatedObject(objectName, resp.VersionId || ''); | ||
| const result = { | ||
| err: null as string | null, | ||
| stdout: JSON.stringify(resp), | ||
| serverSideEncryption: resp.ServerSideEncryption, | ||
| sseKmsKeyId: resp.SSEKMSKeyId, | ||
| }; | ||
| this.setResult(result); | ||
| } catch (error: unknown) { | ||
| const err = error as Error & { name: string; message: string }; | ||
| this.setResult({ | ||
| err: `${err.name}: ${err.message}`, | ||
| stdout: '', | ||
| }); | ||
| } finally { | ||
| client.destroy(); | ||
| } | ||
| }, | ||
| ); | ||
|
|
||
| Then('the PutObject response should have SSE algorithm {string} and KMS key {string}', | ||
| function (this: Zenko, expectedAlgo: string, expectedKey: string) { | ||
| const result = this.getResult() as { | ||
| serverSideEncryption?: string; sseKmsKeyId?: string; | ||
| }; | ||
| if (expectedAlgo) { | ||
| assert.strictEqual(result.serverSideEncryption, expectedAlgo, | ||
| `PutObject SSE: expected "${expectedAlgo}", got "${result.serverSideEncryption}"`); | ||
| } else { | ||
| assert.strictEqual(result.serverSideEncryption, undefined, | ||
| `PutObject SSE: expected absent, got "${result.serverSideEncryption}"`); | ||
| } | ||
| if (expectedKey === 'absent') { | ||
| assert.strictEqual(result.sseKmsKeyId, undefined, | ||
| `PutObject: SSEKMSKeyId should be absent, got "${result.sseKmsKeyId}"`); | ||
| } else { | ||
| assert.ok(result.sseKmsKeyId, 'PutObject: SSEKMSKeyId should be present'); | ||
| } | ||
| }, | ||
| ); | ||
|
|
||
| Then('the GetObject should return the uploaded body with SSE algorithm {string} and KMS key {string}', | ||
| async function (this: Zenko, expectedAlgo: string, expectedKey: string) { | ||
| const bucket = this.getSaved<string>('bucketName'); | ||
| const objectName = this.getSaved<string>('objectName'); | ||
| const expectedBody = this.getSaved<string>('objectBody'); | ||
| const client = buildS3Client(); | ||
| try { | ||
| const resp = await client.send( | ||
| new GetObjectCommand({ Bucket: bucket, Key: objectName }), | ||
| ); | ||
| const body = await resp.Body!.transformToString(); | ||
| assert.strictEqual(body, expectedBody, 'GetObject: body content mismatch'); | ||
|
|
||
| if (expectedAlgo) { | ||
| assert.strictEqual(resp.ServerSideEncryption, expectedAlgo, | ||
| `GetObject SSE: expected "${expectedAlgo}", got "${resp.ServerSideEncryption}"`); | ||
| } else { | ||
| assert.strictEqual(resp.ServerSideEncryption, undefined, | ||
| `GetObject SSE: expected absent, got "${resp.ServerSideEncryption}"`); | ||
| } | ||
| if (expectedKey === 'absent') { | ||
| assert.strictEqual(resp.SSEKMSKeyId, undefined, | ||
| `GetObject: SSEKMSKeyId should be absent, got "${resp.SSEKMSKeyId}"`); | ||
| } else if (expectedKey === 'generated') { | ||
| assert.ok(resp.SSEKMSKeyId, 'GetObject: SSEKMSKeyId should be present'); | ||
| assert.match(resp.SSEKMSKeyId, /^[a-f0-9]{64}$/, | ||
| `GetObject: expected a generated hex key, got "${resp.SSEKMSKeyId}"`); | ||
| } else { | ||
| assert.strictEqual(resp.SSEKMSKeyId, expectedKey, | ||
| `GetObject: expected key "${expectedKey}", got "${resp.SSEKMSKeyId}"`); | ||
| } | ||
| } finally { | ||
| client.destroy(); | ||
| } | ||
| }, | ||
| ); | ||
|
|
||
| Then('it should fail with error {string}', | ||
| function (this: Zenko, expectedError: string) { | ||
| const result = this.getResult(); | ||
| assert.ok(result.err?.includes(expectedError), | ||
| `Expected error "${expectedError}" but got: ${result.err}`); | ||
| }, | ||
| ); | ||
Uh oh!
There was an error while loading. Please reload this page.