e2e test: cluster re-forms after full simultaneous pod restart (stale-address heal)

[matka12]

Follow-up from #333 (requested by @jdheyburn in the #333 review).

#333 adds a reconcile phase that heals a persistent cluster whose members all restarted with new pod IPs (#275): stale peer entries are detected by node ID against the live scraped state and re-introduced with CLUSTER MEET.

This issue tracks an e2e test covering that path, to be added once #333 merges:

1. Create a ValkeyCluster (3 shards, 1 replica, persistence enabled, appendonly).
2. Wait Ready; write a marker key.
3. kubectl delete pod --all in the namespace — all pods restart simultaneously, every pod IP changes while each nodes.conf still holds the old peer IPs.
4. Assert: a StaleAddressesHealed event is emitted, the CR returns to Ready with cluster_state:ok within a bounded window (~1 min), and the marker key is still readable — without any manual CLUSTER MEET.

Optionally also assert the negative: no NodeForgetFailed/forget of live members during recovery (the #333 forget guard).

I'll pick this up after #333 lands.

[jdheyburn]

@matka12 #333 is merged in, are you still able to pick this up?

[matka12]

@jdheyburn yes — opened #380 with the e2e test. It runs the full #275 scenario (persistent 3-shard cluster, canary key, delete all pods at once) and asserts the StaleAddressesHealed event, unassisted return to Ready, cluster_state:ok with the canary intact, and no StaleNodeForgotten during recovery. Verified the exact flow against an image built from current main before pushing.