We are looking for a Big Data Developer (Snowflake).
What will you do?
Migration of existing data warehouses from Azure SQL Database to Snowflake. The current data warehouse, in addition to standard analytical functions, is also a data source for the web application.
Data warehouse implemented in the lambda architecture, combining Real Time data from IoT devices and data from other sources such as SQL server databases. This project is the beginning of the implementation of Single Source Of Truth for the entire organization
This is an early phase of the project, starting implementation after the design phase.
● analyze and organize raw data
● evaluate business needs and objectives
● data modeling
● architecture designing
● developing ELT logic
● ensuring data quality
● code optimization
● preparing documentation
What will you do?
Migration of existing data warehouses from Azure SQL Database to Snowflake. The current data warehouse, in addition to standard analytical functions, is also a data source for the web application.
Data warehouse implemented in the lambda architecture, combining Real Time data from IoT devices and data from other sources such as SQL server databases. This project is the beginning of the implementation of Single Source Of Truth for the entire organization
This is an early phase of the project, starting implementation after the design phase.
● analyze and organize raw data
● evaluate business needs and objectives
● data modeling
● architecture designing
● developing ELT logic
● ensuring data quality
● code optimization
● preparing documentation
Requirements:
2 years of experience developing big data platform.
2 years of experience in Snowflake – Mandatory
Azure knowledge (must have):
o Azure Data Factory
o ADLS gen2
o Event Hub/ IoT Hub/ Stream Analytics
o Networking in Azure (Vnet/ subnets/ gateway/ private endpoints) e. Roles and security (Microsoft Entra ID, security groups etc) – Advantage
o Azure Data Explorer (KQL language) will be an additional advantage
Languages:
o Advanced SQL knowledge (query optimization, partitioning, clustering etc) -> TSQL, ANSI SQL
o Regular Python knowledge
o Git knowledge
o Basic knowledge about NoSQL databases
Developing ADX – Advantage
Data Warehousing knowledge designing, data modelling, methodologies (like Kimball)
Data Lake and Lakehouse concept medallion architecture etc
Big Data concepts knowledge Hive, Spark, partitioning, scaling up and out, streaming processing
Basic DevOps knowledge:
o Azure data DevOps (Boards, tasks, creating PR…)
o CICD processes basic knowledge
2 years of experience developing big data platform.
2 years of experience in Snowflake – Mandatory
Azure knowledge (must have):
o Azure Data Factory
o ADLS gen2
o Event Hub/ IoT Hub/ Stream Analytics
o Networking in Azure (Vnet/ subnets/ gateway/ private endpoints) e. Roles and security (Microsoft Entra ID, security groups etc) – Advantage
o Azure Data Explorer (KQL language) will be an additional advantage
Languages:
o Advanced SQL knowledge (query optimization, partitioning, clustering etc) -> TSQL, ANSI SQL
o Regular Python knowledge
o Git knowledge
o Basic knowledge about NoSQL databases
Developing ADX – Advantage
Data Warehousing knowledge designing, data modelling, methodologies (like Kimball)
Data Lake and Lakehouse concept medallion architecture etc
Big Data concepts knowledge Hive, Spark, partitioning, scaling up and out, streaming processing
Basic DevOps knowledge:
o Azure data DevOps (Boards, tasks, creating PR…)
o CICD processes basic knowledge
This position is open to all candidates.
