You will design, implement, test, deploy, and maintain production-grade data products – pipelines, transformation layers, data quality and reliability systems – using tools like DBT (on Spark) and Databricks. Youll apply best practices in Python and SQL to build scalable and maintainable data transformations, and leverage technologies like LLMs and GenAI to create innovative solutions for real business problems.
This role is ideal for someone who wants technical leadership responsibilities in an AI-first engineering culture – we use LLMs, GenAI, and AI-native development tools as core parts of our daily workflow.
Key Responsibilities
Act as a technical leader within the team – raise engineering standards, drive strong architectural choices, and improve how we build
Own data products end-to-end: design, development, deployment, monitoring, and iteration
Work closely with senior leadership to translate strategic goals into scalable data solutions
Develop and maintain production ETL/ELT pipelines using DBT (on Spark) and orchestrated workflows in Databricks
Build monitoring, alerting, and testing pipelines to ensure reliability and performance in production
Evaluate and introduce new technologies – including AI-native development tools – and integrate the ones that create real impact
Collaborate with customers and external data providers – gathering requirements and making product decisions.
Mentor team members through code reviews, pairing, and knowledge sharing
Must haves
4+ years of experience in production-level data engineering or similar roles
Deep proficiency in SQL and Python
Proven track record of owning and scaling production-grade data pipelines, including versioning, testing, and monitoring
Strong understanding of data modeling, normalization/denormalization trade-offs, and data quality management
Experience with the modern data stack: DBT, Databricks, Spark, Delta Lake
Strong analytical skills – ability to design and evaluate data-driven hypotheses and KPIs
Product and business awareness – you think about the impact of what you build, not just the implementation
Preferred Qualifications
Experience with GenAI and LLM applications – particularly extracting structure from unstructured data at scale
Experience working with external data sources and vendors
Familiarity with Unity Catalog and data governance at scale
Familiarity with Terraform or similar infrastructure-as-code tools
Experience with cost optimization on Databricks (DBU analysis, cluster policies)
Familiarity with cloud-native platforms (AWS preferred)
BSc/BA in Computer Science, Engineering, or a related technical field – or graduation from a top-tier IDF tech unit
