Required Data Infrastructure Engineer
Job Description:
Design, build, and maintain scalable data solutions and infrastructure to support data processing, data quality and data tools for different data professionals and consumers in the organization
Develop and maintain generic processes to ingest data automatically from various sources into our data lake/data warehouse
Lead end to end projects in the data infrastructure group, working closely with users and leaders from the Data Engineering Guild
Collaborate with cross-functional infrastructure teams to integrate data engineering solutions into our products and services to create high-quality deliverables
Stay up-to-date with the latest trends and technologies in data engineering and recommend best practices to improve our data infrastructure.
Job Description:
Design, build, and maintain scalable data solutions and infrastructure to support data processing, data quality and data tools for different data professionals and consumers in the organization
Develop and maintain generic processes to ingest data automatically from various sources into our data lake/data warehouse
Lead end to end projects in the data infrastructure group, working closely with users and leaders from the Data Engineering Guild
Collaborate with cross-functional infrastructure teams to integrate data engineering solutions into our products and services to create high-quality deliverables
Stay up-to-date with the latest trends and technologies in data engineering and recommend best practices to improve our data infrastructure.
Requirements:
3+ years of experience as a Data Engineer or Backend Developer with a focus on Python, Spark/Flink/Kafka, and experience cloud architecture (AWS services, k8s and more)
You are experienced in writing complex SQL queries and data pipelines for big data processing
You are an independent, self-learner who is passionate about data and can translate business and technical needs into data models or services
You have strong problem-solving skills and the ability to work independently
Youre a team player with excellent communication skills
Experience with Airflow, Great Expectations, Open Metadata, data governance practices or any data solutions would be an advantage
Experience building microservices in Python using FastAPI, Flask or similar would also be an advantage.
3+ years of experience as a Data Engineer or Backend Developer with a focus on Python, Spark/Flink/Kafka, and experience cloud architecture (AWS services, k8s and more)
You are experienced in writing complex SQL queries and data pipelines for big data processing
You are an independent, self-learner who is passionate about data and can translate business and technical needs into data models or services
You have strong problem-solving skills and the ability to work independently
Youre a team player with excellent communication skills
Experience with Airflow, Great Expectations, Open Metadata, data governance practices or any data solutions would be an advantage
Experience building microservices in Python using FastAPI, Flask or similar would also be an advantage.
This position is open to all candidates.
