Repository navigation
Preserve native payload storage across write-policy changes - #1224
Conversation
rmcdaniel
left a comment
There was a problem hiding this comment.
Security contract review HOLD on exact test-only head 9d763ba244de3e358064f2cda5d3ecd93508ccda.
The disable/read/prune and disabled-namespace-deletion regressions point in the right direction, but the location-change test encodes the wrong settled product boundary. It requires 409 external_payload_storage_in_use until every existing registration is released. The later owning decision on #546 is explicit: already-stored references keep their original location, while disable/move/prefix changes apply to new writes. A retained object therefore must not pin the namespace's write policy or prevent an administrator from moving future writes.
Before production implementation, replace that refusal contract with a causal two-generation test:
- upload and retain object A under the original local binding;
- successfully move the namespace policy while A still exists;
- verify A remains readable from the original binding;
- verify a new object B is written only to the replacement binding;
- prune/delete A through its original binding without touching B, then prove both registrations and objects reconcile;
- cover namespace deletion when retained objects span both bindings.
The source contract also still needs the issue's remaining security/error seams:
- an original binding that can no longer be resolved must return a specific typed reason, not
external_payload_unavailable/ “temporarily unavailable”; - preserve namespace ownership and refuse cross-namespace reference/policy substitution;
- bind the storage policy/version needed for each registration rather than deriving old-object authority from the mutable current namespace policy. Do not copy credential values into object rows; retain only the provider/location identity or credential reference needed by the supported driver;
- add a concurrent policy-change/upload control proving a write is wholly registered under exactly one immutable policy generation, and that disabling still prevents every new offload.
The current runtime_external_payloads registration records the URI but not a policy generation, while fetch and retention currently construct the driver from the mutable namespace policy. The implementation must close both reads and deletion/reclamation for the original generation. This review does not select a schema or implementation, but the tests must express the settled behavior before they can serve as acceptance evidence.
|
One additional implementation requirement before the schema shape freezes: existing registrations and policy-update races need an explicit binding transition, not only the new-write path.
These are source/upgrade requirements only; the existing contract HOLD remains until the corrected tests and implementation are concrete. |
|
One more consequence of allowing location changes while retained objects remain: cleanup plans must be per-object binding, not one namespace policy per prune. The current Please make each target carry its own immutable binding and stage each This is part of the existing contract HOLD, not a new product scope. |
|
The same per-binding correction must include expired unclaimed cleanup. Resolve the storage binding from each candidate row, preserve the object/backup-hold locks, and add an A+B cleanup pass proving both expired objects are reclaimed through their own roots. Missing original binding should increment the bounded failure/blocked accounting with the typed reason; it must not fall back to B. |
|



Disabling uploads or moving a namespace's write policy currently strands previously offloaded payloads. This native local-storage fix keeps each object's original namespace, storage generation and canonical root, so retained reads, run pruning, abandoned-upload cleanup and namespace deletion continue against the original location. New uploads must still pass the current enabled policy; a write already admitted may finish at its recorded location.
New schema steps add an internal binding and policy generation on SQLite, PostgreSQL, MySQL and MariaDB while preserving every historical bootstrap checksum. Older registrations are bound only when an admitted policy reproduces their recorded URI, including before a policy replacement. Missing or substituted roots refuse without creating replacement directories. Per-object deletion journals preserve mixed generations, backup holds and other namespaces' ownership of the same physical object. The public reference shape is unchanged.
Local validation: 545 library tests passed (98 declared backend/subprocess ignores); all 51 external-payload HTTP cases passed on SQLite, MySQL 8.0, MariaDB 10.11 and PostgreSQL 17. Seventeen backend upgrade/takeover cases passed, including both MySQL/MariaDB interrupted ALTER states, frozen-layout takeover and pending deletion journals. The physical-schema comparators validate both native columns exactly before comparing every remaining PHP column, index and constraint; two local source tests passed 97 assertions including changed-type/default/nullability, duplicate/missing-column and unrelated-drift controls. All 63 current-head checks, merge-queue checks and all four actual-main workflows passed. The merge tree exactly equals qualified head
1c4fecd1c2dc6923ccd6767d9bdb12285bd1fc14; accepted main ise363a472dd9183a66d31e1554824eaceb595253b. No published-artifact or release qualification is claimed.Fixes #546.
The parity contract preserves published PHP’s disabled-policy refusal and requires native retention to delete through its original binding, then report
run_not_foundfor the already pruned run. The two retention steps and three namespace-deletion steps, plus the expected 404 when re-enabling that already deleted namespace, are the only native expectation changes; frozen PHP setup and step values are unchanged. The focused 39 retention, namespace-deletion and native-divergence controls pass; earlier discovery controls also pass. The new schema tests are explicitly selected by the required PHPUnit workflow (9 local tests / 230 assertions), and its existing exact Composer command inventory is rebound without changing execution guards. The final hosted checks passed, including the corrected native retention and namespace-deletion expectations. Topic refs are absent and the worktree is clean.