Lead Big-Data Engineer for a a well-known financial company
Hybrid Position
We are looking for a Lead Data Engineer to join our CTO Developer Experiences Platform Team to lead the implementation of our vision to build Data Lake House & MLOps on the AWS cloud.
some of the things you do on a day-to-day basis:
– Plan and build Data Lake House in the AWS cloud based on future technologies most companies will use in only 3-5 years.
– Have end to end ownership: Design, develop, deploy, measure and maintain our data platform.
– Solve production issues in the framework code.
– Collaborate with 40+ data engineers/scientists, understand their functional needs and wants and build products to accelerate their productivity and quality of work.
– Influence directly on the way millions of people consume insurance and financial services.
– Research and build Generative AI services and applications.
– Lead and mentor data engineers, providing guidance and fostering a culture of continuous learning and professional development.
– Stay up to date with emerging technologies, industry trends, and best practices related to new technology, and proactively recommend their adoption when appropriate.
– Communicate complex architectural concepts and recommendations effectively to both technical and non-technical stakeholders.
– Help the in analyzing and adopting new technologies including assessing and embedding startup technologies.
Our Tech Stack:
Python, Java, Spark, Kafka, Parquet, IceBerg, Avro, AWS S3, AWS Glue, AWS Athena, AWS DataZone, AWS Lake Formation, AWS SageMaker, AWS Document DB, AWS ElasticCahe, AWS OpenSearch, AWS DRS, AWS QuickSight, AWS EKS, AWS Managed Airflow.
