Cost-Effective Big Data Orchestration Using Dagster: A Multi-Platform Approach
TLDR: Spark-based data PaaS solutions are convenient. But they come with their own set of challenges such as a high vendor lock-in and obscured costs. We show how to use a dedicated orchestrator (dagster-pipes). It can not only make Databricks an implementation detail but also save cost. Also, it improves developer productivity. It allows you to take back control.
Sep 2, 2024
The diaspora model for human migration
May 20, 2024
Visual analytics of mobility network changes observed using mobile phone data during COVID-19 pandemic
Feb 8, 2023
Viral variant-resolved wastewater surveillance of SARS-CoV-2 at national scale
Jul 16, 2022
Specialization in Criminal Careers
Jul 16, 2022
Identifying the root cause of cable network problems with machine learning
Mar 14, 2022
Data Anonymization – The key to innovation
This is based on the previous post Can you tell the nuts & berries apart in each group? The official Link to Manz is: https://www.manz.at/produkte/zeitschriften/ecolex/archiv#02-2022 https://rdb.manz.at/document/rdb.tso.LIecolex20220253
Mar 1, 2022
Mobility changes in Austria in fall 2021
Find the full analysis here: https://www.csh.ac.at/lockdown-for-unvaccinated-mobility-in-austria/ and here: https://www.csh.ac.at/wp-content/uploads/2021/11/2021-11-26-CSH-Policy-Brief-Mobilitat-Herbst-2021-final.pdf
Nov 26, 2021
Monitoring supply networks from mobile phone data for estimating the systemic risk of an economy
Figure description: (a) Probability $p(s|c)$ to find a supply link, sij , given that there exists a communication link, cij, between firms i and j for communication links exceeding a given call duration, dij. Error bars denote the quartiles of a bootstrap simulation described in SI Text 1.
Oct 13, 2021
Preprint: Varieties of mobility measures: Comparing survey and mobile phone data during the COVID-19 pandemic
This publication is currently still a preprint under review.
Oct 1, 2021