We ingest our data from our operational DBs, telematics devices, and more, working with several data types (both structured and unstructured). Our challenge is to provide building tools and infrastructure to empower other teams leveraging data-mesh concepts.
In this role youll:
Help build Lemonades data platform, designing and implementing data solutions for all application requirements in a distributed microservices environment
Build data-platform ingestion layers using streaming ETLs and Change Data Capture
Implement pipelines and scheduling infrastructures
Ensure compliance, data-quality monitoring, and data governance on data platform
Implement large-scale batch and streaming pipelines with data processing frameworks
Collaborate with other Data Engineers, Developers, BI Engineers, ML Engineers, Data Scientists, Analysts and Product managers
Share knowledge with other team members and promote engineering standards
5+ years of prior experience as a data engineer or data infra engineer
B.S. in Computer Science or equivalent field of study
Knowledge of databases (SQL, NoSQL)
Proven success in building large-scale data infrastructures such as Change Data Capture, and leveraging open source solutions such as Airflow & DBT, building large-scale streaming pipelines, and building customer data platforms
Experience with Python, Pulumi\Terraform, Apache Spark, Snowflake, AWS, K8s, Kafka
Ability to work in an office environment a minimum of 3 days a week
Enthusiasm about learning and adapting to the exciting world of AI a commitment to exploring this field is a fundamental part of our culture
