Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion docs/book/src/examples/cni-readiness.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,36 @@ To test this, add a new node to the cluster.
Once Calico starts, the reporter will update it to `True`.

3. **Check Taint Removal**:
As soon as the condition becomes `True`, the Node Readiness Controller will remove the taint, and workloads will be scheduled.
As soon as the condition becomes `True`, the Node Readiness Controller will remove the taint, and workloads will be scheduled.

## Cilium CNI Integration & Bootstrap Taints

When using **Cilium** as your CNI plugin:

1. **Align Bootstrap Taints**: Configure Cilium's agent not-ready taint flag to use the standardized `readiness.k8s.io/*` prefix:
```yaml
# Helm values for Cilium
extraArgs:
- --agent-not-ready-taint-key=readiness.k8s.io/network-not-ready
```

2. **Bootstrap NodeReadinessRule**: Once Cilium initializes networking, it sets the built-in Kubernetes node condition `NetworkUnavailable=False`. You can define a rule to untaint nodes based on `NetworkUnavailable`:
```yaml
# cilium-network-readiness-rule.yaml
apiVersion: readiness.node.x-k8s.io/v1alpha1
kind: NodeReadinessRule
metadata:
name: cilium-network-readiness-rule
spec:
conditions:
- type: "NetworkUnavailable"
requiredStatus: "False"
taint:
key: "readiness.k8s.io/network-not-ready"
effect: "NoSchedule"
value: "true"
enforcementMode: "bootstrap-only"
```

> [!NOTE]
> Cilium updates `NetworkUnavailable=False` during node bootstrap. For continuous post-startup health monitoring, pair this with a custom DaemonSet probe and `enforcementMode: "continuous"`.
4 changes: 4 additions & 0 deletions examples/cni-readiness/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,7 @@ This example demonstrates how to use the Node Readiness Controller to ensure nod
3. The `NodeReadinessRule` (`network-readiness-rule.yaml`) instructs the controller to remove the startup taint once the `projectcalico.org/CalicoReady` condition becomes `True`.
4. The reporter is deployed with `hostNetwork: true` to reach Calico's local health endpoint.
5. The reporter needs a dedicated ServiceAccount (`cni-reporter`) with permissions to patch node status.

### Cilium Integration:
For Cilium CNI, configure `--agent-not-ready-taint-key=readiness.k8s.io/network-not-ready` and apply `cilium-network-readiness-rule.yaml` targeting `NetworkUnavailable=False`.

17 changes: 17 additions & 0 deletions examples/cni-readiness/cilium-network-readiness-rule.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
apiVersion: readiness.node.x-k8s.io/v1alpha1
kind: NodeReadinessRule
metadata:
name: cilium-network-readiness-rule
spec:
conditions:
- type: "NetworkUnavailable"
requiredStatus: "False"
taint:
key: "readiness.k8s.io/network-not-ready"
effect: "NoSchedule"
value: "true"
enforcementMode: "bootstrap-only"
nodeSelector:
matchExpressions:
- key: node-role.kubernetes.io/control-plane
operator: DoesNotExist