Overview

In this Lead Data Engineer role, you will own end-to-end data pipelines across Bronze, Silver and Gold layers, enabling scalable data products for the Procurement domain. You will drive DevOps-centric deployment practices in Azure DevOps and GitHub, collaborating with modelers, stewards, and business partners to deliver production-grade solutions. You’ll work with dbt, Databricks, PySpark, and CI/CD automation to shape data architecture and governance at scale. This position offers leadership opportunities, cross-functional impact, and exposure to cutting-edge data engineering practices in a general setting.

Compensaciones / Beneficios

- growth opportunities
- global teamwork
- culture of collaboration
- learning and development
- voice and influence in shaping future
- opportunity to impact millions

Responsabilidades

- Design and implement end-to-end data pipelines for ingestion, transformation, and storage across Bronze, Silver, and Gold layers
- Develop and manage data ingestion processes from source systems via ETL and API-based methods
- Build modular transformation frameworks using dbt, PySpark, and SQL with staging models and load metadata in alignment with Data Vault 2.0
- Design, build, and maintain CI/CD and DevOps processes in Azure DevOps and GitHub Actions with automated testing and governance
- Implement IaC and deployment automation for data platform resources (dbt, Databricks, clusters, dependencies)




- Establish monitoring, observability, and operational excellence for pipeline reliability and rapid issue resolution
- Drive data quality, governance, and compliance with automated testing, lineage, and FAIR data principles
- Provide technical leadership and cross-functional collaboration across data engineers, modelers, stewards, BI developers, data scientists, and business SMEs

Requisitos principales

- Strong technical expertise in dbt, SQL, Python, Spark (PySpark), Databricks, Git, Azure DevOps, and GitHub
- Proven experience designing and maintaining production-grade data pipelines (5+ years)
- Deep CI/CD and automation experience with Azure DevOps Pipelines and GitHub Actions; IaC knowledge (Terraform, Bicep) is a plus
- Advanced Git and repository management; governance across Azure DevOps and GitHub
- Cloud data engineering experience (Azure) with Databricks jobs, clusters, and workflows
- Extensive knowledge of Data Vault 2.0 architecture (Raw Vault, Business Vault, Gold-layer models)
- Experience with data ingestion patterns (batch ETL, incremental, CDC, API-based ingestion)
- Strong focus on data governance and quality; exposure to scientific datasets is a plus
- Leadership mindset with ability to lead teams, manage projects, and collaborate across functions and geographies

- leadership
- collaboration
- curiosity
- dbt
- SQL
- Python

📌 Lead Data Engineer (Cross Domain) - dsm-firmenich (Barcelona)
🏢 dsm-firmenich
📍 Barcelona