CLDSRV-892: store part checksum in part shadow object

[leif-scality]

• UploadPart
  • Store part checksum in part shadow objet
  • Verify MPU checksum config with received headers
  • Handle dual checksum case (default MPU checksum + custom client checkum)
• Refactor
  • dataStore now takes checksum object as argument instead of parsing the headers
  • getChecksumDataFromHeaders returns null instead of a default checksum, it is now up to the caller to select a default
• Functional tests will be added by https://scality.atlassian.net/browse/S3C-11120 that will make ListParts return the checksum

[bert-e]

Hello leif-scality,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options

name	description	privileged	authored
/after_pull_request	Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval	Bypass the pull request author's approval	⭐
/bypass_build_status	Bypass the build and test status	⭐
/bypass_commit_size	Bypass the check on the size of the changeset TBA	⭐
/bypass_incompatible_branch	Bypass the check on the source branch prefix	⭐
/bypass_jira_check	Bypass the Jira issue check	⭐
/bypass_peer_approval	Bypass the pull request peers' approval	⭐
/bypass_leader_approval	Bypass the pull request leaders' approval	⭐
/approve	Instruct Bert-E that the author has approved the pull request.		✍️
/create_pull_requests	Allow the creation of integration pull requests.
/create_integration_branches	Allow the creation of integration branches.
/no_octopus	Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity	Change review acceptance criteria from one reviewer at least to all reviewers
/wait	Instruct Bert-E not to run until further notice.

Available commands

name	description	privileged
/help	Print Bert-E's manual in the pull request.
/status	Print Bert-E's current status in the pull request TBA
/clear	Remove all comments from Bert-E from the history TBA
/retry	Re-start a fresh build TBA
/build	Re-start a fresh build TBA
/force_reset	Delete integration branches & pull requests, and restart merge process from the beginning.
/reset	Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

[bert-e]

Incorrect fix version

The Fix Version/s in issue CLDSRV-892 contains:

• None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 9.4.0

Please check the Fix Version/s of CLDSRV-892, or the target
branch of this pull request.

[codecov]

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
8839	1	8838	0

View the full list of 1 ❄️ flaky test(s)

"before each" hook for "should retrieve a part that overwrote another part originally copied from an MPU"::GET object With default signature With PartNumber field uploadPartCopy overwrite "before each" hook for "should retrieve a part that overwrote another part originally copied from an MPU"

Flake rate in main: 100.00% (Passed 0 times, Failed 8 times)

Stack Traces | 6.3s run time

Connection timed out after 5000 ms

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[claude]

• Bug: null dereference in createAndStoreObject.js:332 — checksum.type = 'FULL_OBJECT' crashes when the md-only path passes null as checksum (line 291: next(null, dataGetInfo, _md5, null)). Wrap in if (checksum) guard.
  • Guard with if (checksum) { checksum.type = 'FULL_OBJECT'; }
• Test quality: require() inside it() callbacks in objectPutPartChecksum.js — The same require('validateChecksums') is repeated at lines 193, 223, and 259 inside test functions.
  • Move to the top of the file with the other requires

Review by Claude Code

[claude]

• Crash bug: createAndStoreObject.js:333 — checksum.type = 'FULL_OBJECT' throws TypeError when checksum is null (md-only path passes null at line 291)
  • Guard with if (checksum) before setting .type
• Missing error handler wrapping: prepareStream.js:117 — default/UNSIGNED-PAYLOAD branch does not wrap errCb with jsutil.once() when secondary is present, unlike V4 and trailer cases
  • Wrap with jsutil.once(errCb) for consistency
• require() inside test callbacks: tests/unit/api/objectPutPartChecksum.js:192,222,258 — require() calls should be at the top of the file
  • Move to top-level require block
• Missing .catch(done) on Promise chains: tests/unit/api/objectPutPartChecksum.js:197,227,261 — unhandled rejections will cause test hangs
  • Add .catch(done) to each Promise chain

Review by Claude Code

[claude]

LGTM

Review by Claude Code

[claude]

LGTM

The primary/secondary checksum architecture is clean. All callers of dataStore and prepareStream are updated with the new checksums parameter. The getChecksumDataFromHeaders returning null instead of the default is handled correctly at every call site with || defaultChecksumData. Stream error guards (jsutil.once) are properly applied when secondary introduces a third error source. Trailer wiring in the STREAMING-UNSIGNED-PAYLOAD-TRAILER path correctly routes to the secondary stream only when present, and falls back to the primary when there is no secondary. Test coverage is thorough across unit and functional layers, covering single-checksum, dual-checksum, algo mismatch rejection, and trailer flows.

Review by Claude Code

[claude]

LGTM

The refactor cleanly separates checksum parsing (caller) from stream setup (prepareStream), and the dual-checksum pipeline for MPU parts is well-designed: secondary upstream for validation, primary downstream for storage. Stream error handling correctly uses jsutil.once guards when multiple transforms are piped. The checksum algorithm mismatch rejection logic in objectPutPart correctly allows any algo for default MPUs while rejecting mismatches for explicitly configured ones. Test coverage is thorough across unit and functional tests.

Review by Claude Code