Skip to content

Commit a5291a0

Browse files
peterschmidt85peterschmidt85
committed
[Blog] Orchestrating GPUs on DigitalOcean and AMD Developer Cloud
1 parent 4d858ae commit a5291a0

4 files changed

Lines changed: 153 additions & 7 deletions

File tree

‎docs/assets/stylesheets/extra.css‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1318,10 +1318,6 @@ html .md-footer-meta.md-typeset a:is(:focus,:hover) {
13181318
display: none;
13191319
}
13201320

1321-
.md-tabs__item:nth-child(7) {
1322-
display: none;
1323-
}
1324-
13251321
/* .md-tabs__item:nth-child(5):after {
13261322
content: url('data:image/svg+xml,<svg fill="black" xmlns="http://www.w3.org/2000/svg" width="20px" height="20px" viewBox="0 0 16 16"><polygon points="5 4.31 5 5.69 9.33 5.69 2.51 12.51 3.49 13.49 10.31 6.67 10.31 11 11.69 11 11.69 4.31 5 4.31" data-v-e1bdab2c=""></polygon></svg>');
13271323
line-height: 14px;

‎docs/blog/posts/digitalocean-and-amd-dev-cloud.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
---
2+
title: Orchestrating GPUs on DigitalOcean and AMD Developer Cloud
3+
date: 2025-09-04
4+
description: "TBA"
5+
slug: digitalocean-and-amd-dev-cloud
6+
image: https://dstack.ai/static-assets/static-assets/images/digitalocean-and-amd-dev-cloud.png
7+
categories:
8+
- Changelog
9+
---
10+
11+
# Orchestrating GPUs on DigitalOcean and AMD Developer Cloud
12+
13+
Orchestration automates provisioning, running jobs, and tearing them down. While Kubernetes and Slurm are powerful in their domains, they lack the lightweight, GPU-native focus modern teams need to move faster.
14+
15+
`dstack` is built entirely around GPUs. Our latest update introduces native integration with [DigitalOcean :material-arrow-top-right-thin:{ .external }](https://www.digitalocean.com/products/gradient/gpu-droplets){:target="_blank"} and
16+
[AMD Developer Cloud :material-arrow-top-right-thin:{ .external }](https://www.amd.com/en/developer/resources/cloud-access/amd-developer-cloud.html){:target="_blank"}, enabling teams to provision cloud GPUs and run workloads more cost-efficiently.
17+
18+
<img src="https://dstack.ai/static-assets/static-assets/images/digitalocean-and-amd-dev-cloud.png" width="630"/>
19+
20+
<!-- more -->
21+
22+
## About Digital Ocean
23+
24+
DigitalOcean is one of the leading cloud platforms offering GPUs both as VMs and as bare-metal clusters equipped with NVIDIA and AMD GPUs.
25+
26+
## About AMD Developer Cloud
27+
28+
AMD Developer Cloud is a new cloud platform designed to make AMD GPUs easily accessible to developers, academics, open-source contributors, and AI innovators worldwide.
29+
30+
## Why dstack
31+
32+
Unlike Kubernetes, dstack provides a high-level, AI-engineer-friendly interface where GPUs work out of the box—no custom operators or low-level setup required. Unlike Slurm, it’s use-case agnostic, equally suited for training, inference, benchmarking, and dev environments.
33+
34+
With the new DigitalOcean and AMD Developer Cloud backends, you can now provision NVIDIA or AMD GPU VMs and run workloads with a single CLI command.
35+
36+
## Getting started
37+
38+
Best part about `dstack` is that it's very easy to get started.
39+
40+
1. Create a project in Digital Ocean or AMD Developer Cloud
41+
2. Get credits or approve a payment method
42+
3. Create an API key
43+
44+
Then, configure the backend in `~/.dstack/server/config.yml`:
45+
46+
<div editor-title="~/.dstack/server/config.yml">
47+
48+
```yaml
49+
projects:
50+
- name: main
51+
backends:
52+
- type: amddevcloud
53+
project_name: my-amd-project
54+
creds:
55+
type: api_key
56+
api_key: ...
57+
```
58+
59+
</div>
60+
61+
For DigitalOcean, set `type` to `digitalocean`.
62+
63+
Install and start the `dstack` server:
64+
65+
<div class="termy">
66+
67+
```shell
68+
$ pip install "dstack[server]"
69+
$ dstack server
70+
```
71+
72+
</div>
73+
74+
For more details, see [Installation](../../docs/installation/index.md).
75+
76+
Use the `dstack` CLI to
77+
manage [dev environments](../../docs/concepts/dev-environments.md), [tasks](../../docs/concepts/tasks.md),
78+
and [services](../../docs/concepts/services.md).
79+
80+
<img src="https://dstack.ai/static-assets/static-assets/images/amd-dev-cloud.png" width="750"/>
81+
82+
The `digitalocean` and `amddevcloud` backends support NVIDIA and AMD GPU VMs, respectively, and allow you to run
83+
[dev environments](../../docs/concepts/dev-environments.md) (interactive development), [tasks](../../docs/concepts/tasks.md)
84+
(training, fine-tuning, or other batch jobs), and [services](../../docs/concepts/services.md) (inference).
85+
86+
Here’s an example of a service configuration:
87+
88+
<div editor-title="examples/modes/gpt-oss/120b.dstack.yml">
89+
90+
```yaml
91+
type: service
92+
name: gpt-oss-120b
93+
94+
model: openai/gpt-oss-120b
95+
96+
env:
97+
- HF_TOKEN
98+
- MODEL=openai/gpt-oss-120b
99+
# To enable AITER, set below to 1. Otherwise, set it to 0.
100+
- VLLM_ROCM_USE_AITER=1
101+
# To enable AITER Triton unified attention
102+
- VLLM_USE_AITER_UNIFIED_ATTENTION=1
103+
# below is required in order to enable AITER unified attention by disabling AITER MHA
104+
- VLLM_ROCM_USE_AITER_MHA=0
105+
image: rocm/vllm-dev:open-mi300-08052025
106+
commands:
107+
- |
108+
vllm serve $MODEL \
109+
--tensor-parallel $DSTACK_GPUS_NUM \
110+
--no-enable-prefix-caching \
111+
--disable-log-requests \
112+
--compilation-config '{"full_cuda_graph": true}'
113+
port: 8000
114+
115+
volumes:
116+
# Cache downloaded models
117+
- /root/.cache/huggingface:/root/.cache/huggingface
118+
119+
resources:
120+
gpu: MI300X:8
121+
shm_size: 32GB
122+
```
123+
124+
</div>
125+
126+
As with any configuration, you can apply it via `dstack apply`. If needed, `dstack` will automatically provision new VMs and run the inference endpoint.
127+
128+
<div class="termy">
129+
130+
```shell
131+
$ dstack apply -f examples/models/gpt-oss/120b.dstack.yml
132+
133+
# BACKEND RESOURCES PRICE
134+
1 amddevcloud (alt1) cpu=20 mem=240GB disk=720GB MI300X:192GB:8 $15.92
135+
136+
Submit the run? [y/n]:
137+
```
138+
139+
</div>
140+
141+
> If you prefer to use bare-metal clusters with `dstack`, you can create an [SSH fleet](../../docs/concepts/fleets.md#ssh-fleets).
142+
> This way, you’ll be able to run [distributed tasks](../../docs/concepts/tasks.md#distributed-tasks) efficiently across the cluster.
143+
144+
!!! info "What's next?"
145+
1. Check [Quickstart](../../docs/quickstart.md)
146+
2. Learn more about [DigitalOcean :material-arrow-top-right-thin:{ .external }](https://www.digitalocean.com/products/gradient/gpu-droplets){:target="_blank"} and
147+
[AMD Developer Cloud :material-arrow-top-right-thin:{ .external }](https://www.amd.com/en/developer/resources/cloud-access/amd-developer-cloud.html){:target="_blank"}
148+
3. Explore [dev environments](../../docs/concepts/dev-environments.md),
149+
[tasks](../../docs/concepts/tasks.md), [services](../../docs/concepts/services.md),
150+
and [fleets](../../docs/concepts/fleets.md)
151+
4. Join [Discord :material-arrow-top-right-thin:{ .external }](https://discord.gg/u8SmfwPpMd){:target="_blank"}

examples/llms/gpt-oss/amd/120b.dstack.yml renamed to examples/models/gpt-oss/amd/120b.dstack.yml

File renamed without changes.

‎mkdocs.yml‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -294,11 +294,10 @@ nav:
294294
- Tenstorrent: examples/accelerators/tenstorrent/index.md
295295
- Models:
296296
- Wan2.2: examples/models/wan22/index.md
297-
- Changelog: blog/changelog.md
298-
- Case studies: blog/case-studies.md
299-
- Benchmarks: blog/benchmarks.md
300297
- Blog:
301298
- blog/index.md
299+
- Case studies: blog/case-studies.md
300+
- Benchmarks: blog/benchmarks.md
302301
# - Discord: https://discord.gg/u8SmfwPpMd" target="_blank
303302
# - Changelog: https://github.com/dstackai/dstack/releases" target="_blank
304303
# - GitHub: https://github.com/dstackai/dstack" target="_blank

0 commit comments

Comments
 (0)