Back to catalog
dstack
MPL-2.0dstack is an open-source alternative to Kubernetes for AI workloads, automating multi-cloud GPU provisioning and model serving within private VPCs.
Open-source alternative to — compare it against:
Overview
dstack is an open-source control plane for provisioning and running machine learning workloads across cloud providers and local hardware. It standardizes GPU allocation through declarative YAML manifests, eliminating custom provisioning scripts and vendor wrappers. Models and training data remain inside your own infrastructure accounts, avoiding the exposure risks of shared multi-tenant platforms.
Why dstack instead of Kubernetes
Kubernetes is the standard foundation for containerized software, but adapting it for AI introduces heavy operational friction. Running ML on Kubernetes requires GPU device plugins, storage drivers, and complex operators. For teams focused on training and inference, managing cluster machinery creates unnecessary overhead. dstack orchestrates at the compute level: it provisions spot or on-demand instances directly through cloud APIs or SSH nodes, executing workloads and terminating compute automatically when jobs finish.
The core advantage for security-conscious teams is sovereign infrastructure control. Rather than routing weights and datasets through a third-party SaaS plane, dstack executes runs entirely inside your own cloud VPCs or on-premises servers. You retain custody over cloud credentials, storage buckets, and network boundaries. Where dstack yields to Kubernetes is general application hosting; it focuses on batch tasks, model endpoints, and dev workspaces rather than general microservice fleets. Transitioning to dstack means replacing verbose Helm charts with compact YAML manifests, shedding cluster maintenance without sacrificing isolation.
What it does well
- Multi-cloud GPU provisioning: requests accelerators by type and memory in YAML, querying backends to secure available spot or on-demand instances across providers.
- Declarative workflow manifests: versions tasks, endpoints, and dev workspaces in Git alongside application code.
- Ephemeral dev environments: spins up remote workspaces accessible via SSH or IDEs, with idle shutdown rules to curb spend.
- Heterogeneous accelerator support: runs workloads on NVIDIA, AMD, Google TPU, and Tenstorrent hardware without custom driver management scripts.
- Secure model serving: routes inference requests with token authentication and TLS, supporting prefill-decode disaggregation for high-throughput serving.
- SSH fleet integration: pools bare-metal servers and on-premises workstations into unified clusters without requiring Kubernetes.
