Dagster-slurm: Bringing Modern Data Orchestration to Slurm-Managed
dagster-slurm is an open-source Python integration that allows data scientists and research software engineers to run Dagster pipeline assets on both a laptop and Slurm-managed HPC …
Discovering the SUPER in computing - dagster-slurm for reproducible research on HPC
Dagster is a modern data orchestrator that emphasises reproducibility, observability, and a strong developer experience. In parallel, most high-performance computing (HPC) centres …
Scaling Data Pipelines @Magenta Telekom
Making humans and machines collaborate efficiently.
Simple Sovereign Scalable Data Stack
**Duck taming** the tigers of distributed systems. Join us for a practical talk on building a fast, portable, and sovereign data & AI stack with: DuckDB, Dagster, Ray, Small …
Simple Sovereign Scalable Data Stack
Tired of cloud lock-in and surprise bills? This talk shows how to build a fast, portable analytics stack around DuckDB and Dagster. Along the way of our journey to sovereignty and …
Scaling data pipelines (Big Data Minds conference Frankfurt)
Datenabhängigkeiten, Tool-Silos und Entwicklungsprozesse müssen keine Hindernisse sein – sie können gezielt zu einem integralen Bestandteil einer leistungsfähigen Datenplattform …
Scaling data pipelines @Magenta Telekom
Tackling data challenges via the orchestrator.
Scaling data pipelines @Telekom
Tackling data challenges via the orchestrator.
Migrating to rattler-build
Sharing my experience migrating from conda-build to rattler-build for python noarch packages.