Key Responsibilities
Maintain and develop Enterprise grade Data Lake and Data Warehouse environments
Create data infrastructure for various R&D groups across the organization that will assist them in building and optimizing our products into an innovative industry leader.
Work with Data Experts to assist with data-related technical issues and support their data infrastructure needs.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources on top of Azure cloud.
Create and maintain optimal data pipeline architecture
B.Sc. in Computer Science, Computer Engineering or a similar degree.
3+ years of experience as a Software or Data Engineer working on production systems.
Proven experience in Apache Spark using Python or Scala.
Advanced working SQL knowledge and experience working with relational databases.
Experience building, optimizing and automating, data pipelines, architectures and data sets.
Strong project management and organizational skills.
Advantage – Azure cloud Services: Storage Accounts, EventHub, DataBricks.
