We are looking for a Senior Data Platform Engineer with +6 years of experience to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. The professional will own and evolve the foundations that data product teams build on: Apache Airflow orchestration (migrating from ADF), Azure resource management (Key Vault, identity and access, networking, storage), Azure DevOps CI/CD and Infrastructure as Code, Databricks and Unity Catalog governance, observability and cost efficiency, while remaining hands?Responsibilities:Orchestration Platform (Apache Airflow & ADF)Own the Apache Airflow platform: environments, deployment, upgrades, configuration, connections, variables, pools and DAG standards . Work on the migration from Azure Data Factory to Airflow and the evolution of the Metadata-Driven Orchestration Framework (DAG generation, reusable operators, dependency and scheduling patterns).

Ensure Airflow is reliable and observable : SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.Azure Resource ManagementManage Azure resources with Infrastructure as Code (Terraform) : Databricks workspaces, Data Lake Storage, Data Factory, Key Vault, networking and supporting services.Key Vault, service principals and managed identities, secret scopes and rotation.Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following least privilege, plus access requests and periodic reviews.Own Azure DevOps : repositories, CI/CD pipelines, branch policies and multi?environment promotion, including deployment validation and Terraform state.Track cost and governance of Azure resources: tagging, budgets,



right?Hands?on Data Platform evolutionContribute to the implementation of the Data Catalog and Data Quality Framework and it's lineage across the platform.Unity Catalog structure, cluster and job policies, Auto Loader ingestion patterns, and performance and cost optimization of workloads.making, stating trade offs (performance, cost, reliability, complexity) with data.Collaborate with Architects, Product Owners and data engineers in technical maturation of data products.line support to the Run/Operations team for complex production topics.Troubleshoot and resolve platform incidents, failures and performance issues; lead root cause analysis and post?mortems.Continuously improve runbooks, monitoring and operational processes .

Technical

Skills6+ years of experience in data engineering, with significant time on platform or infrastructure topicsStrong technical expertise with Python and SQL (advanced), including PySpark and Delta Lake semanticsDeep hands?on experience with Apache Airflow in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloadsHands?on experience with Azure resource management :Key Vault, managed identities and service principals, secret managementAccess management : Azure RBAC, Entra ID, Unity Catalog permissionsAzure DevOps (repos, pipelines, service connections, environments) and CI/CD for data platformsADLS, networking and security basics,



Azure Data FactoryInfrastructure as Code (Terraform) and multi?environment promotionHands?on experience with Databricks and Unity Catalog (compute, jobs, governance)Observability and monitoring, alerting and incident managementSolid understanding of Data Lakehouse architecture, Medallion architecture and metadata?functional teams and to influence without authorityOwnership mindset and attention to detailGood communication skills with both technical and business stakeholders: leads with the recommendation, quantifies, and is transparent about uncertaintyWhat we're looking for in you:Led an Airflow implementation or migration (ideally from ADF), or run it at scale.driven orchestration or reusable ETL frameworks.Applied FinOps practices on Azure and Databricks: cost attribution, budgets, forecasting and measurable savingsExperience running Airflow on Kubernetes or operating containerized workloads on AKSWhy you'll love working with usAt Capitole, we believe our people are our greatest strength. 1,200/year for your professional development (trainings, events, certifications...)Monthly feedback sessions with your team to keep learning and growingPrivate health insurance fully covered by CapitolePayflow benefits (meal, transport, childcare vouchers)Wellhub subscriptionDiscounts with top brands through Capitole ClubLife at CapitoleTeam buildings every two months (our summer party and Christmas dinner are legendary!)capitole-consulting.Glassdoor ReviewsWill have access to confidential information related to Capitole and the project they are working on.Must comply with the security and internal policies of both the company and the client.

📌 Hibrido Bcn - Senior Data Platform Engineer (Barcelona)
🏢 Capitole Consulting
📍 Barcelona