feature(eviction): add event when EvictPod failed

[googs1025]

/kind feature

• add event when EvictPod failed.
• add unit test coverage

When it comes to eviction, what we really care about are the errors that occur during evictions. So I tend to include events when an eviction fails. Besides metrics and application logs, events are also a valuable source of information. Typically, operations personnel respond to events to manage the cluster effectively.

[googs1025]

/retest

[ingvagabund]

What about to pass k8serrors.XXX directly? I.e.

[]struct {
	description string
	node        *v1.Node
	evictedPod  *v1.Pod
	pods        []runtime.Object
	wantErr     error
}{
	...
	{
		description: "test pod eviction - pod absent (not found error)",
		node:        node1,
		evictedPod:  pod1,
		pods:        []runtime.Object{test.BuildTestPod("p2", 400, 0, "node1", nil), test.BuildTestPod("p3", 450, 0, "node1", nil)},
		wantErr:  k8serrors.NewNotFound(v1.Resource("pods"), pod1.Name),
	},
}
...
fakeClient.PrependReactor("create", "pods/eviction", func(action core.Action) (handled bool, ret runtime.Object, err error) {
	return true, nil, test.wantErr
})

The original wantErr bool can be turned into wantErr != nil.

[googs1025]

Because we return using the fmt.Error, we cannot directly use k8serrors.XXX, but we can simplify it to directly use fmt.Error to return

descheduler/pkg/descheduler/evictions/evictions.go

Lines 213 to 221 in da52983

	err := client.PolicyV1().Evictions(eviction.Namespace).Evict(ctx, eviction)
	if apierrors.IsTooManyRequests(err) {
	return fmt.Errorf("error when evicting pod (ignoring) %q: %v", pod.Name, err)
	}
	if apierrors.IsNotFound(err) {
	return fmt.Errorf("pod not found when evicting %q: %v", pod.Name, err)
	}
	return err

[ingvagabund]

The whole function body can be reduced to return true, nil, test.expectedError.

[googs1025]

done

[googs1025]

/test pull-descheduler-test-e2e-k8s-master-1-30

[ingvagabund]

If these events get produced by default in a cluster with many pods the cluster gets flooded with events. This needs to be disabled by default and enabled if needed. By extending DeschedulerConfiguration with a new field.

[googs1025]

Ok, this sounds more reasonable. I will add a new field in the DeschedulerConfiguration. Because we have similar monitoring cluster-specific failure events (ex: scheduling failure, eviction failure, etc.), this can quickly let cluster administrators know important event information.

[googs1025]

i will handle this in weekend

[googs1025]

done

[ingvagabund]

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ingvagabund

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [ingvagabund]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment