Ecosystem integrations

[aojea]

Some projects it will be nice to get integrations

• MPI operator https://github.com/kubeflow/mpi-operator
  • https://github.com/google/dranet/blob/main/site/content/docs/user/mpi-operator.md
• Slurm on Kube https://github.com/slinkyproject
  • Expose affinity mask Numa
  • Expose fabric topology https://slurm.schedmd.com/topology.html?
• Kube Ray https://github.com/ray-project/kuberay
  • reference [Core] InfiniBand and RDMA support for Ray object store ray-project/ray#30094
  • "Ray Job Yaml Deployment Example" with only a few GPUs and NICs and running in parallel
  • Kuberay #162
• Jobset https://github.com/kubernetes-sigs/jobset
• Kueue https://github.com/kubernetes-sigs/kueue
• GKE https://cloud.google.com/ai-hypercomputer/docs/create/gke-ai-hypercompute-custom
• Flux operator https://github.com/flux-framework/flux-operator @vsoch
• XPK [Feature] Integrate DraNet AI-Hypercomputer/xpk#482
• NVidia GPU driver https://github.com/NVIDIA/k8s-dra-driver-gpu
  • Add docs and example manifest on how to use NVIDIA DRA with DRANet #127
• VLLM https://github.com/vllm-project/vllm
• pytorch example https://docs.pytorch.org/docs/stable/distributed.html
• GKE TPUs Trillium
• Kubeflow

[aojea]

tagging @siyuanfoundation for kuberay and @akulkapoor-google for mpi-operator

[mwysokin]

I'd like to propose Kueue as another potential integration. Especially since there's a high chance that today it won't play nice with Kueue's Topology Aware Scheduling.

[vsoch]

We can definitely add support for the Flux Operator! How are the devices (?) exposed to the applications in a pod?

[siyuanfoundation]

/cc @kevin85421 for kuberay

[kevin85421]

ray-project/ray#52938 Ray recently merged the first POC PR to natively support collective communication backends. We start from NCCL and GLOO and then we will also support other backends like RDMA and UCX later.

[kevin85421]

I am currently working on KubeRay release. I will explore it after I finish the release process ray-project/kuberay#3739.

[kannon92]

@aojea Before we ask for community input, is there consideration about moving this to a vendor neutral location?

I think this would be a good candidate for kubernetes-sigs/dranet?

Its not officially supported by google so not sure if it should live in the google org.

[aojea]

@aojea Before we ask for community input, is there consideration about moving this to a vendor neutral location?

I think this would be a good candidate for kubernetes-sigs/dranet?

Its not officially supported by google so not sure if it should live in the google org.

It's being under consideration, we are gathering positive feedback like this and I'm working on it

[kannon92]

I haven't looked over the code but there isn't anything here particular to google right?

I could use this with on-prem?

[aojea]

I haven't looked over the code but there isn't anything here particular to google right?

I could use this with on-prem?

yes, the code is structured since the beginning to not be vendor specific

[kannon92]

And do you know what problems we would need to solve if we wanted to use CRIO?