Skip to content

Feat(numa cpu pressure eviction config) add upper and lower bound #209

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cheney-lin merged 1 commit into from
Jul 20, 2026
Merged

Feat(numa cpu pressure eviction config) add upper and lower bound #209

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,14 @@ spec:
format: int64
minimum: 0
type: integer
lowerBoundRatio:
description: |-
LowerBoundRatio is the soft threshold ratio (0~1) applied to the NUMA CPU
usage ratio. When enough recent samples cross this bound but the upper bound
is not yet reached, the plugin reports SOFT_MET together with a NodeCondition
maximum: 1
minimum: 0
type: number
metricRingSize:
description: MetricRingSize is the size of the metric
ring buffer for calculating NUMA CPU pressure.
Expand All @@ -288,6 +296,15 @@ spec:
maximum: 1
minimum: 0
type: number
upperBoundRatio:
description: |-
UpperBoundRatio is the hard threshold ratio (0~1) applied to the NUMA CPU
usage ratio. When enough recent samples cross this bound, the plugin reports
HARD_MET together with a NodeCondition + NoSchedule taint, which allows the
eviction manager to actually evict candidate pods.
maximum: 1
minimum: 0
type: number
type: object
numaSysCPUPressureEvictionConfig:
description: NumaSysCPUPressureEvictionConfig holds configurations
Expand Down
21 changes: 18 additions & 3 deletions pkg/apis/config/v1alpha1/adminqos.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -554,18 +554,15 @@ type NumaCPUPressureEvictionConfig struct {
// +kubebuilder:validation:Minimum=0
// +optional
GracePeriod *int64 `json:"gracePeriod,omitempty"`

// ThresholdExpandFactor expands the metric threshold from a specific machine to set the eviction threshold.
// E.g., 1.1 means a 10% increase.
// +kubebuilder:validation:Minimum=0
// +optional
ThresholdExpandFactor *float64 `json:"thresholdExpandFactor,omitempty"`

// CpuUsageRatioThreshold is the CPU usage ratio threshold for NUMA-level CPU pressure eviction.
// +kubebuilder:validation:Minimum=0
// +optional
CpuUsageRatioThreshold *float64 `json:"cpuUsageRatioThreshold,omitempty"`

// CandidateCount is the candidate pod count when selecting pods to be evicted.
// +kubebuilder:validation:Minimum=0
// +optional
Expand All @@ -574,6 +571,24 @@ type NumaCPUPressureEvictionConfig struct {
// SkippedPodKinds is the pod kind that will be skipped when selecting pods to be evicted.
// +optional
SkippedPodKinds []string `json:"skippedPodKinds,omitempty"`

// UpperBoundRatio is the hard threshold ratio (0~1) applied to the NUMA CPU
// usage ratio. When enough recent samples cross this bound, the plugin reports
// HARD_MET together with a NodeCondition + NoSchedule taint, which allows the
// eviction manager to actually evict candidate pods.
// +kubebuilder:validation:Minimum=0
// +kubebuilder:validation:Maximum=1
// +optional
UpperBoundRatio *float64 `json:"upperBoundRatio,omitempty"`

// LowerBoundRatio is the soft threshold ratio (0~1) applied to the NUMA CPU
// usage ratio. When enough recent samples cross this bound but the upper bound
// is not yet reached, the plugin reports SOFT_MET together with a NodeCondition
// + NoSchedule taint, which only prevents new scheduling without evicting pods.
// +kubebuilder:validation:Minimum=0
// +kubebuilder:validation:Maximum=1
// +optional
LowerBoundRatio *float64 `json:"lowerBoundRatio,omitempty"`
Comment on lines +575 to +591

@coderabbitai coderabbitai Bot Jul 17, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== locate relevant symbols =="
rg -n "UpperBoundRatio|LowerBoundRatio|validation:.*(UpperBoundRatio|LowerBoundRatio)|CEL|XValidation|cross-field|upper.*lower|lower.*upper" \
  pkg/apis/config/v1alpha1 config/crd/bases -S

echo
echo "== file size =="
wc -l pkg/apis/config/v1alpha1/adminqos.go config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml

echo
echo "== ast-grep outline for adminqos.go =="
ast-grep outline pkg/apis/config/v1alpha1/adminqos.go --view expanded | sed -n '1,220p'

echo
echo "== relevant CRD excerpt around lower/upper =="
sed -n '240,330p' config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml

Repository: kubewharf/katalyst-api

Length of output: 27476

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== search for cross-field validation annotations =="
rg -n "XValidation|x-kubernetes-validations|kubebuilder:validation:.*(XValidation|oneOf|anyOf)|rule:" pkg/apis/config/v1alpha1 config/crd/bases -S

echo
echo "== inspect load and usage bound pairs in adminqos.go =="
sed -n '424,450p' pkg/apis/config/v1alpha1/adminqos.go
echo
sed -n '807,845p' pkg/apis/config/v1alpha1/adminqos.go

echo
echo "== inspect corresponding CRD excerpts =="
sed -n '188,230p' config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml
echo
sed -n '378,430p' config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml

Repository: kubewharf/katalyst-api

Length of output: 212

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== cross-field validation annotations (if any) =="
rg -n "XValidation|x-kubernetes-validations|kubebuilder:validation:.*(oneOf|anyOf|allOf)|rule:" \
  pkg/apis/config/v1alpha1 config/crd/bases -S || true

echo
echo "== NUMA and system CPU pressure snippets =="
sed -n '498,555p' pkg/apis/config/v1alpha1/adminqos.go
echo
sed -n '807,845p' pkg/apis/config/v1alpha1/adminqos.go

echo
echo "== generated CRD snippets =="
sed -n '250,310p' config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml
echo
sed -n '404,435p' config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml

Repository: kubewharf/katalyst-api

Length of output: 10468

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== usages of upper/lower bound ratios in repo =="
rg -n "UpperBoundRatio|LowerBoundRatio|loadUpperBoundRatio|loadLowerBoundRatio|usageUpperBoundRatio|usageLowerBoundRatio" -S .

echo
echo "== inspect likely implementation file(s) if present =="
fd -a -e go . | rg "evict|pressure|qos|adminqos|numa|load|usage" | head -n 50

Repository: kubewharf/katalyst-api

Length of output: 8094

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "== read the utility helpers =="
sed -n '1,260p' pkg/utils/adminqos.go

echo
echo "== check for ratio-related logic in eviction helpers =="
sed -n '1,260p' pkg/utils/eviction.go

echo
echo "== any cross-field checks in API package =="
rg -n ">=|<=|lower.*upper|upper.*lower|compare|validate|normalize" pkg/apis/config/v1alpha1 pkg/utils -S

Repository: kubewharf/katalyst-api

Length of output: 3244

Enforce lower ≤ upper for these ratio fields.
LowerBoundRatio can still exceed UpperBoundRatio here, and the same gap exists for the load/usage upper/lower pairs in pkg/apis/config/v1alpha1/adminqos.go. Add cross-field validation and regenerate config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml so invalid threshold order is rejected at admission.

📍 Affects 2 files
  • pkg/apis/config/v1alpha1/adminqos.go#L536-L552 (this comment)
  • config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml#L262-L270
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/apis/config/v1alpha1/adminqos.go` around lines 536 - 552, Add cross-field
validation enforcing each lower threshold to be less than or equal to its
corresponding upper threshold in the AdminQOS configuration, including the ratio
fields and all load/usage upper/lower pairs in AdminQOSConfiguration. Update the
API validation markers and regenerate
config/crd/bases/config.katalyst.kubewharf.io_adminqosconfigurations.yaml so
admission rejects invalid threshold ordering; the CRD site requires only the
generated validation output.

}

// NumaSysCPUPressureEvictionConfig holds the configurations for NUMA-level system CPU pressure eviction.
Expand Down
10 changes: 10 additions & 0 deletions pkg/apis/config/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.