Vinted Standardizes Data Pipelines With DAG Generator
Vinted migrated its data infrastructure to the cloud and built a DAG generator to standardize large-scale decentralized pipelines during the migration. The generator reads the dbt manifest and unfolds model-level lineage into task-per-model Airflow DAGs, enabling ExternalTaskSensor-based cross-domain dependencies across ~20 teams. This reduces costly reruns, improves visibility, and preserves team ownership while automating fine-grained handoffs.
Key Points
- 1Unfolds dbt manifest into task-per-model Airflow DAGs across ~20 teams
- 2Reduces cross-domain ambiguity and failed reruns by adding model-level visibility
- 3Enables sensors and ExternalTaskSensor-based dependencies to automate fine-grained cross-team handoffs
Scoring Rationale
Practical, organization-scale engineering solution with strong operational benefits; limited novelty beyond combining established dbt and Airflow capabilities.
Sources
Public references used for this report.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.
Try 250 free problems
