Capitole is growing fast, and we want you to be part of the journey!We are looking for aSenior Data Platform Engineerwith+6 years of experienceto help drive the evolution of ourAzure-based Data PlatformandLakehouse architecture. This role is focused on theplatform itself. The professional will own and evolve thefoundationsthat data product teams build on:Apache Airflow orchestration (migrating from ADF) ,Azure resource management(Key Vault, identity and access, networking, storage),Azure DevOps CI/CD and Infrastructure as Code ,Databricks and Unity Catalog governance ,observabilityandcost efficiency, while remaininghands‑onand close to the teams that use the platform.Responsibilities:Orchestration Platform (Apache Airflow & ADF)Own theApache Airflowplatform: environments, deployment, upgrades, configuration, connections, variables, pools andDAG standards .Work on the migration fromAzure Data Factory to Airflowand the evolution of theMetadata-Driven Orchestration Framework(DAG generation, reusable operators, dependency and scheduling patterns).Ensure Airflow isreliable and observable : SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.Azure Resource ManagementManage Azure resources withInfrastructure as Code (Terraform) : Databricks workspaces,Data Lake Storage, Data Factory,Key Vault, networking and supporting services.Ownsecrets and identity management : Key Vault, service principals and managed identities, secret scopes and rotation.Ownaccess management : Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, followingleast privilege, plus access requests and periodic reviews.OwnAzure DevOps : repositories,CI/CD pipelines, branch policies andmulti‑environment promotion, including deployment validation and Terraform state.Trackcost and governanceof Azure resources: tagging, budgets, right‑sizing and cost attribution (e.G. query tagging).Hands‑on Data Platform evolutionContribute to the implementation of theData CatalogandData Quality Frameworkand it’slineageacross the platform.Driveobservability : pipelines that are observable, debuggable and alert‑driven, with clear ownership of failures.Operate theDatabricksplatform:Unity Catalogstructure,



cluster and job policies,Auto Loaderingestion patterns, and performance and cost optimization of workloads.Technical LeadershipParticipate inarchitecture reviewsand technical decision‑making, statingtrade offs(performance, cost, reliability, complexity) with data.Define and promoteengineering standards : naming conventions, patterns and guidelines, through code reviews and design reviews.Collaborate with Architects, Product Owners and data engineers intechnical maturationof data products.Build and maintaindeveloper tooling and automation, includingAI‑assisted engineeringworkflows (shared agent instructions, skills, automated PR review).Support & Operations (Run Team Collaboration)Providesecond‑line supportto the Run/Operations team for complex production topics.Troubleshoot and resolveplatform incidents, failures and performance issues; leadroot cause analysisand post‑mortems.Ensure properhandover, documentation and operational readinessof platform changes.Continuously improverunbooks, monitoring and operational processes .Requirements:Technical Skills6+ years of experience in data engineering, with significant time onplatform or infrastructuretopicsStrong technical expertise with Python and SQL (advanced), including PySpark andDelta LakesemanticsDeep hands‑on experience withApache Airflowin production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloadsHands‑on experience withAzure resource management :Key Vault, managed identities and service principals, secret managementAccess management : Azure RBAC, Entra ID, Unity Catalog permissionsAzure DevOps(repos, pipelines, service connections, environments) and CI/CD for data platformsADLS, networking and security basics, Azure Data FactoryInfrastructure as Code(Terraform)



and multi‑environment promotionHands‑on experience withDatabricksandUnity Catalog(compute, jobs, governance)Observability and monitoring, alerting and incident managementSolid understanding of Data Lakehouse architecture,Medallion architectureandmetadata‑driven architecture ; awareness ofcost managementand performanceSoft SkillsStrong problem‑solving and analytical thinkingAbility to work in cross‑functional teams and toinfluence without authorityOwnership mindset and attention to detailGood communication skills with both technical and business stakeholders: leads with the recommendation, quantifies, and is transparent about uncertaintyWhat we’re looking for in you:Led anAirflowimplementation or migration (ideally from ADF), or run it at scale.Builtmetadata‑driven orchestrationor reusable ETL frameworks.Built developer tooling orLLM-agent workflows(MCP, skills, automated review) for engineering teams.Applied FinOps practices on Azure and Databricks: cost attribution, budgets, forecasting and measurable savingsExperience running Airflow on Kubernetes or operating containerized workloads on AKSWhy you’ll love working with usAt Capitole, we believe our people are our greatest strength. That's why we offer:€1,200/year for your professional development (trainings, events, certifications…)Monthly feedback sessions with your team to keep learning and growingPrivate health insurance fully covered by CapitolePayflow benefits (meal, transport, childcare vouchers)Wellhub subscriptionDiscounts with top brands through Capitole ClubLife at CapitoleTeam buildings every two months (oursummer partyand Christmas dinner are legendary!)Football, padel, and basketball teams sponsored by CapitoleTech communities where sharing knowledge is part of our DNAAnd above all… anAMAZING TEAMthat you’ll love being a part of!Want to learn more about us?capitole-consulting.ComSee what our teams are saying about us: Glassdoor ReviewsWill have access to confidential information related to Capitole and the project they are working on.Must comply with the security and internal policies of both the company and the client.Must sign an NDA (Non-Disclosure Agreement).#J-18808-Ljbffr

📌 Hibrido Bcn - Senior Data Platform Engineer (Madrid)
🏢 Capitole Consulting
📍 Madrid