fix: add validation for rejecting operator owned keys - #399
Conversation
Signed-off-by: Deepak Punjabi <deepakpunjabi13@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Included review availability: Your plan provides up to 2 included reviews per hour; 1 remains after this review. 📝 WalkthroughWalkthroughThe API and CRD now reject operator-managed Valkey configuration keys, including case variants, and limit ChangesReserved configuration validation
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
| Filename | Overview |
|---|---|
| api/v1alpha1/valkeycluster_types.go | Adds a synchronized reserved-key list and case-insensitive field-level CEL validation for cluster configuration. |
| config/crd/bases/valkey.io_valkeyclusters.yaml | Publishes the generated validation rule and bounded map schema in the installed CRD. |
| internal/controller/config_reserved_keys_test.go | Verifies consistency among the operator-generated base configuration, Go reserved-key list, and CEL literal. |
| internal/controller/valkeycluster_config_validation_test.go | Exercises the new admission contract through API-server create and update requests, including valid and case-insensitive cases. |
Reviews (1): Last reviewed commit: "fix: add validation for rejecting operat..." | Re-trigger Greptile
|
Ran this on a k3d cluster against a live API server rather than only through the suite, since a CEL rule is only as good as what the apiserver actually admits. Rejection works for The part worth adding to the description is the upgrade path, because it is better than a reader would assume and nothing here says so. I created a cluster carrying
Existing clusters therefore keep reconciling and get nudged on every unrelated edit rather than being wedged. Worth stating, since "operator-owned keys are now rejected" reads much scarier than that. One observation that is not about this PR. On that: |
| "dir", | ||
| "port", | ||
| "protected-mode", | ||
| "shutdown-on-sigterm", |
There was a problem hiding this comment.
We could allow users to override this default. But I would say let's wait for the issue to come in first requesting the override so that we can understand the use case.
| // It is set high deliberately, because raising a bound later is backwards compatible while lowering one locks out anyone already above it. | ||
| // Valkey has roughly 200 directives in total, so 1000 cannot realistically be reached. | ||
| // +kubebuilder:validation:MaxProperties=1000 | ||
| // +kubebuilder:validation:XValidation:rule="self.all(key, !(key.lowerAscii() in ['aclfile','cluster-allow-replica-migration','cluster-config-file','cluster-enabled','cluster-node-timeout','cluster-replica-validity-factor','dir','port','protected-mode','shutdown-on-sigterm','tls-auth-clients','tls-ca-cert-file','tls-cert-file','tls-cluster','tls-key-file','tls-port','tls-replication']))",message="spec.config must not set operator-owned keys (aclfile, cluster-allow-replica-migration, cluster-config-file, cluster-enabled, cluster-node-timeout, cluster-replica-validity-factor, dir, port, protected-mode, shutdown-on-sigterm, tls-auth-clients, tls-ca-cert-file, tls-cert-file, tls-cluster, tls-key-file, tls-port, tls-replication): the operator sets these itself and a user value would be ignored or would break its connection to the nodes" |
There was a problem hiding this comment.
Instead of listing these out twice, could we link to documentation where folks can go to find the blocked configs?
You would also need to add the documentation, which was going to be another comment I had :) Perhaps in docs/valkeycluster.md under config.
| "tls-key-file", | ||
| "tls-port", | ||
| "tls-replication", | ||
| } |
There was a problem hiding this comment.
Can you also add these, which are managed but applied directly on the pod.
cluster-announce-ip
primaryuser
primaryauth
This PR closes #393.
Summary
Previously, setting options in
ValkeyCluster.spec.configthat the operator manages internally caused a silent mismatch. Because Valkey uses the last entry in its configuration file, the operator's default settings silently overwrote user-defined settings without showing an error.When TLS was disabled, the operator didn't override
portortls-settings. A user could accidentally change or close the port the operator needed to connect, breaking the deployment.Features / Behaviour Changes
spec.confignow rejects below operator-owned keys(case-insensitive):aclfilecluster-allow-replica-migrationcluster-config-filecluster-enabledcluster-node-timeoutcluster-replica-validity-factordirportprotected-modeshutdown-on-sigtermtls-auth-clientstls-ca-cert-filetls-cert-filetls-clustertls-key-filetls-porttls-replicationCluster directives the operator does not set(
cluster-require-full-coverage,cluster-migration-barrierandcluster-allow-reads-when-down) stay available.Rule is applied on create and update both.
Implementation
api/v1alpha1/valkeycluster_types.goaddsReservedConfigKeysand a field-level CEL rule onConfig.Two new test files.
CRD is regenerated.
spec.configsetsmaxProperties: 1000to accept the CRD. Valkey has ~200 directives as of now.config_reserved_keys_test.goadds test for config drift as config is mentioned in 3 places -getBaseConfig,ReservedConfigKeysand the CEL rule.TestReservedConfigKeysMatchCELRuleextracts the CEL list out ofvalkeycluster_types.goto compare it.Limitations
ValkeyNode.spec.configis not covered in this PR.Testing
config_reserved_keys_test.go, 3 plain Go tests without cluster:TestReservedConfigKeysMatchBaseConfigderives the real key set by callinggetBaseConfigwith TLS both off and on, and asserts it equalsReservedConfigKeys.TestReservedConfigKeysMatchCELRuleparses the CEL literal from source and compares it to the same list.TestReservedConfigKeysAreLowercaseguards the lowercased comparison, since an uppercase entry could never match.valkeycluster_config_validation_test.go, 26 envtest specs against a Kubernetes API server:17 specs, one per reserved key, each asserting rejection. Generated from
ReservedConfigKeysso the coverage cannot fall behind the list.3 specs for case-insensitivity:
Cluster-Enabled,PROTECTED-MODE,TLS-Port.3 specs for ordinary tunables, the cluster directives the operator does not own, and an absent
config.Rejection when a reserved key is mixed in with valid ones.
Rejection when a reserved key is added by update rather than create.
The message names the offending keys, since there is no webhook to elaborate.
make test,make lint,pre-commit run --all-filespasses.Checklist
Before submitting the PR make sure the following are checked:
pre-commit run --all-filesor hooks on commit)