Overview In this Lead Data Engineer role, you will own end-to-end data pipelines across Bronze, Silver and Gold layers, enabling scalable data products for the Procurement domain. You will drive DevOps-centric deployment practices in Azure DevOps and GitHub, collaborating with modelers, stewards, and business partners to deliver production-grade solutions. You'll work with dbt, Databricks, PySpark, and CI/CD automation to shape data architecture and governance at scale.

This position offers leadership opportunities, cross-functional impact, and exposure to cutting-edge data engineering practices in a global setting. Compensaciones / Ventajas growth opportunities global teamwork culture of collaboration learning and development voice and influence in shaping future opportunity to impact millions Responsabilidades Design and implement end-to-end data pipelines for ingestion, transformation, and storage across Bronze, Silver, and Gold layers Develop and manage data ingestion processes from source systems via ETL and API-based methods Build modular transformation frameworks using dbt, PySpark, and SQL with staging models and load metadata in alignment with Data Vault 2.0 Design, build, and maintain CI/CD and DevOps processes in Azure DevOps and GitHub Actions with automated testing and governance Implement IaC and deployment automation for data platform resources (dbt, Databricks, clusters, dependencies)



Establish monitoring, observability, and operational excellence for pipeline reliability and rapid issue resolution Drive data quality, governance, and compliance with automated testing, lineage, and FAIR data principles Provide technical leadership and cross-functional collaboration across data engineers, modelers, stewards, BI developers, data scientists, and business SMEs Requisitos principales Strong technical expertise in dbt, SQL, Python, Spark (PySpark), Databricks, Git, Azure DevOps, and GitHub Proven experience designing and maintaining production-grade data pipelines (5+ years) Deep CI/CD and automation experience with Azure DevOps Pipelines and GitHub Actions; IaC knowledge (Terraform, Bicep) is a plus Advanced Git and repository management; governance across Azure DevOps and GitHub Cloud data engineering experience (Azure) with Databricks jobs, clusters, and workflows Extensive knowledge of Data Vault 2.0 architecture (Raw Vault, Business Vault, Gold-layer models) Experience with data ingestion patterns (batch ETL, incremental, CDC, API-based ingestion) Strong focus on data governance and quality; exposure to scientific datasets is a plus Leadership mindset with ability to lead teams, manage projects, and collaborate across functions and geographies leadership collaboration curiosity dbt SQL Python

📌 Lead Data Engineer (Cross Domain) (Barcelona)
🏢 dsm-firmenich
📍 Barcelona