Lead Big-Data Engineer for a a well-known financial company
Hybrid Position
We are looking for a Lead data Engineer to join our CTO Developer Experiences Platform Team to lead the implementation of our vision to build data Lake House & MLOps on the AWS cloud.
some of the things you do on a day-to-day basis:
Plan and build data Lake House in the AWS cloud based on future technologies most companies will use in only 3-5 years.
Have end to end ownership: Design, develop, deploy, measure and maintain our data platform.
Solve production issues in the framework code.
Collaborate with 40+ data engineers/scientists, understand their functional needs and wants and build products to accelerate their productivity and quality of work.
Influence directly on the way millions of people consume insurance and financial services.
Research and build Generative AI services and applications.
Lead and mentor data engineers, providing guidance and fostering a culture of continuous learning and professional development.
Stay up to date with emerging technologies, industry trends, and best practices related to new technology, and proactively recommend their adoption when appropriate.
Communicate complex architectural concepts and recommendations effectively to both technical and non-technical stakeholders.
Help the in analyzing and adopting new technologies including assessing and embedding startup technologies.
Our Tech Stack: Python, JAVA, Spark, Kafka, Parquet, IceBerg, Avro, AWS S3, AWS Glue, AWS Athena, AWS DataZone, AWS Lake Formation, AWS SageMaker, AWS Document DB, AWS ElasticCahe, AWS OpenSearch, AWS DRS, AWS QuickSight, AWS EKS, AWS Managed Airflow.
Hybrid Position
We are looking for a Lead data Engineer to join our CTO Developer Experiences Platform Team to lead the implementation of our vision to build data Lake House & MLOps on the AWS cloud.
some of the things you do on a day-to-day basis:
Plan and build data Lake House in the AWS cloud based on future technologies most companies will use in only 3-5 years.
Have end to end ownership: Design, develop, deploy, measure and maintain our data platform.
Solve production issues in the framework code.
Collaborate with 40+ data engineers/scientists, understand their functional needs and wants and build products to accelerate their productivity and quality of work.
Influence directly on the way millions of people consume insurance and financial services.
Research and build Generative AI services and applications.
Lead and mentor data engineers, providing guidance and fostering a culture of continuous learning and professional development.
Stay up to date with emerging technologies, industry trends, and best practices related to new technology, and proactively recommend their adoption when appropriate.
Communicate complex architectural concepts and recommendations effectively to both technical and non-technical stakeholders.
Help the in analyzing and adopting new technologies including assessing and embedding startup technologies.
Our Tech Stack: Python, JAVA, Spark, Kafka, Parquet, IceBerg, Avro, AWS S3, AWS Glue, AWS Athena, AWS DataZone, AWS Lake Formation, AWS SageMaker, AWS Document DB, AWS ElasticCahe, AWS OpenSearch, AWS DRS, AWS QuickSight, AWS EKS, AWS Managed Airflow.
Requirements:
What will you bring:
Tech Skills:
Previous experience in data engineering and familiarity with AWS databases services like RDS, SPARK, Glue, OpenSearch and more.
5+ years experience developing large scale distributed data systems and services, experience in Big Data tools.
5+ years of working experience with AWS Cloud and services.
5+ years of expertise in the Software design and architecture process. Experience with SQL and No-SQL.
Has at least 3 + years of software or data engineering development experience in one or more of the following programming languages: Python, JAVA, or Scala.
Production systems understanding. Has good understanding and experience of CI/CD practices and Git, including MLOps practices.
What will you bring:
Tech Skills:
Previous experience in data engineering and familiarity with AWS databases services like RDS, SPARK, Glue, OpenSearch and more.
5+ years experience developing large scale distributed data systems and services, experience in Big Data tools.
5+ years of working experience with AWS Cloud and services.
5+ years of expertise in the Software design and architecture process. Experience with SQL and No-SQL.
Has at least 3 + years of software or data engineering development experience in one or more of the following programming languages: Python, JAVA, or Scala.
Production systems understanding. Has good understanding and experience of CI/CD practices and Git, including MLOps practices.
This position is open to all candidates.
