docs: DRA updates for 26.7.0 - #458
Conversation
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
Documentation preview |
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
f2b15a6 to
f2436f2
Compare
| This functionality replaces the traditional GPU allocation method used by the NVIDIA Kubernetes Device Plugin. | ||
| * ComputeDomains: an abstraction for secure `Multi-Node NVLink (MNNVL) <https://docs.nvidia.com/multi-node-nvlink-systems/index.html>`_ for NVIDIA GB200 and similar systems. | ||
| GPU Operator management of DRA is available as a technology preview. | ||
| Technology preview features are not supported in production environments and are not functionally complete. |
There was a problem hiding this comment.
As per our decision, these two lines should be replaced with some version of this:
GPU Operator supports the DRA Driver as a modern alternative to the Device Plugin, but certain features of the DRA Driver are in alpha and not fully supported. As such, the DRA Driver does not currently provide full feature parity with the Device Plugin and it is important that users gain awareness of the features that are not yet implemented or fully supported in the DRA Driver before selecting it for their production needs.
There was a problem hiding this comment.
Changed, but not verbatim. The subject of your text is the software, the subject of my proposal is the reader ("you").
|
|
||
| .. _known-issues: | ||
| The DRA Driver for NVIDIA GPUs and the NVIDIA Kubernetes Device Plugin provide alternative mechanisms for allocating NVIDIA GPU resources. | ||
| The mechanisms do not provide feature parity. |
There was a problem hiding this comment.
I put a version of this above under the important note that replaces the previously discussed Tech Preview statement
There was a problem hiding this comment.
I think the about feature parity is duplicated now, in two back to back sections!
| The mechanisms do not provide feature parity. | ||
| A cluster can have either a ``GPUCluster`` resource for DRA or a ``ClusterPolicy`` resource for the Device Plugin, but not both. | ||
|
|
||
| Use DRA for workloads that have the following requirements: |
There was a problem hiding this comment.
Is this a good approach? Bear in mind that these are not parallel alternatives that depend on user needs, but rather DRA is on track to permanently replace the device plugin, but is still not as mature. If we wanted to use this type of comparison, we would have to focus on still missing features in DRA, and for its alpha features say that use device plugin if you requires supported production usage of the feature, and basically nothing else.
There was a problem hiding this comment.
I thought it was a good approach. My first thought was to focus on the needs of the user and answer "why would I want to use the DRA driver?"
Regarding the "focus on still missing features..." and the result that folks might prefer to use the device plugin--isn't that an honest assessment?
What do you propose? It doesn't have to be my way.
There was a problem hiding this comment.
Yes, that's fine, but the way to think about it, IMO, is that "folks might prefer to use the device plugin" for "a while longer". At some point, device plugin will be deprecated and they will have to move. So giving them information to decide if they are ready to make the switch at this point is a good idea. However, it's a question of when to switch and not if to switch.
There was a problem hiding this comment.
My expectation is that this documentation is gradually updated as the DRA is gradually updated to meet parity. The language gets stronger as development progresses.
There was a problem hiding this comment.
I also want to echo what @bmozaffa is saying and the below bullet points doesn't clearly state key differences from the API, feature and scheduling perspective. I think just the comparison table added below is sufficient for users to make a decision.
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
Signed-off-by: Mike McKiernan <mmckiernan@nvidia.com>
280f6bf to
df69a0b
Compare
| With the DRA Driver for NVIDIA GPUs, your Kubernetes workloads can allocate and consume the following two types of resources: | ||
| GPU Operator supports the DRA Driver as a more recent alternative to the NVIDIA Device Plugin for Kubernetes. | ||
| However, some features of the DRA driver are alpha maturity and not fully supported. | ||
| The driver does not provide feature parity with the device plugin. |
There was a problem hiding this comment.
@mikemckiernan this gives an impression that some key features are missing in the DRA driver. That is actually quite opposite and it has much more advanced and flexible device allocation features. Instead we should just point users to carefully review current limitations {minimum k8s version, feature gate dependencies etc}.
| For example: | ||
| * - Device Injection and Managed Components | ||
| - Requires a Container Device Interface (CDI)-compatible runtime. | ||
| ``GPUCluster`` manages DRA, ComputeDomains, NVIDIA Data Center GPU Manager (DCGM), DCGM Exporter, and the DRA validator. |
There was a problem hiding this comment.
We have to highlight the fact that with GPUCluster the GPU stack is much simpler and manages DRA drivers, DCGM/Exporter and a validation pod.
| * ``NVMLDeviceHealthCheck`` is alpha and disabled by default. | ||
| The gRPC health probe reports kubelet plugin availability, and DCGM provides telemetry. | ||
| Neither provides DRA allocation health status. | ||
| * This release does not provide scheduler-accounted capacity sharing among independent ``ResourceClaim`` objects. |
There was a problem hiding this comment.
the ConsumableShares feature is supported as an alpha like other feature gates and users can explore that just like other features.
3 participants
Primary page for DRA: https://nvidia.github.io/cloud-native-docs/review/pr-458/gpu-operator/latest/dra-intro-install.html