Dagster-slurm: Bringing Modern Data Orchestration to Slurm-Managed
dagster-slurm is an open-source Python integration that allows data scientists and research software engineers to run Dagster pipeline assets on both a laptop and Slurm-managed HPC …
Metaxy + Dagster-Slurm for Efficient Multimodal Pipelines
How to combine Metaxy, Dagster-Slurm, Docling, and Ray to run incremental multimodal pipelines on sovereign AI infrastructure.
Dagster + Slurm
Run and orchestrate Dagster workloads on Slurm-based HPC environments with production-oriented patterns.
•
1 min read
Rediscovering the SUPER in Supercomputing
Improving developer experience on HPC systems with dagster.
