Software Engineers Embrace Data Engineering Career Transition

Adi Polak, Sarah Usher, and Matthias Niehoff discuss transitioning from software engineering to data engineering, drawing on experiences at Confluent, consulting, and industry practice. They emphasize hands-on learning, applying software-engineering principles like testing and CI/CD to data pipelines, and mastering tools such as Spark, streaming systems, and query optimization. The discussion highlights evolving architectures that separate compute and storage and the importance of strong data modeling skills.