Descripción

? Capitole is growing fast, and we want you to be part of the journey!

We are looking for a Senior Data Platform Engineer with 6 years of experience to help drive the evolution of our Azure-based Data Platform and Lakehouse architecture. This role is focused on the platform itself. The professional will own and evolve the foundations that data product teams build on: Apache Airflow orchestration (migrating from ADF), Azure resource management (Key Vault, identity and access, networking, storage), Azure DevOps CI/CD and Infrastructure as Code, Databricks and Unity Catalog governance, observability and cost efficiency, while remaining hands-on and close to the teams that use the platform.

? Responsibilities:

Orchestration Platform (Apache Airflow & ADF)

- Own the Apache Airflow platform: environments, deployment, upgrades, configuration, connections, variables, pools and DAG standards.

- Work on the migration from Azure Data Factory to Airflow and the evolution of the Metadata-Driven Orchestration Framework (DAG generation, reusable operators, dependency and scheduling patterns).

- Ensure Airflow is reliable and observable: SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.

Azure Resource Management

- Manage Azure resources with Infrastructure as Code (Terraform): Databricks workspaces, Data Lake Storage, Data Factory, Key Vault, networking and supporting services.

- Own secrets and identity management: Key Vault, service principals and managed identities, secret scopes and rotation.

- Own access management: Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following least privilege, plus access requests and periodic reviews.

- Own Azure DevOps: repositories, CI/CD pipelines, branch policies and multi-environment promotion, including deployment validation and Terraform state.

- Track cost and governance of Azure resources: tagging, budgets, right-sizing and cost attribution (e.g. query tagging).

Hands-on Data Platform evolution

- Contribute to the implementation of the Data Catalog and Data Quality Framework and it’s lineage across the platform.

- Drive observability: pipelines that are observable, debuggable and alert-driven, with clear ownership of failures.

- Operate the Databricks platform: Unity Catalog structure, cluster and job policies, Auto Loader ingestion patterns,



and performance and cost optimization of workloads.

Technical Leadership

- Participate in architecture reviews and technical decision-making, stating trade-offs (performance, cost, reliability, complexity) with data.

- Define and promote engineering standards: naming conventions, patterns and guidelines, through code reviews and design reviews.

- Collaborate with Architects, Product Owners and data engineers in technical maturation of data products.

- Build and maintain developer tooling and automation, including AI-assisted engineering workflows (shared agent instructions, skills, automated PR review).

Support & Operations (Run Team Collaboration)

- Provide second-line support to the Run/Operations team for complex production topics.

- Troubleshoot and resolve platform incidents, failures and performance issues; lead root cause analysis and post-mortems.

- Ensure proper handover, documentation and operational readiness of platform changes.

- Continuously improve runbooks, monitoring and operational processes.

? Requirements:

Technical Skills

- 6 years of experience in data engineering, with significant time on platform or infrastructure topics

- Strong technical expertise with Python and SQL (advanced), including PySpark and Delta Lake semantics

- Deep hands-on experience with Apache Airflow in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads

- Hands-on experience with Azure resource management:

- Key Vault, managed identities and service principals, secret management

- Access management: Azure RBAC, Entra ID, Unity Catalog permissions

- Azure DevOps (repos, pipelines, service connections, environments) and CI/CD for data platforms

- ADLS, networking and security basics, Azure Data Factory

- Infrastructure as Code (Terraform) and multi-environment promotion

- Hands-on experience with Databricks and Unity Catalog (compute, jobs, governance)

- Observability and monitoring,



alerting and incident management

- Solid understanding of Data Lakehouse architecture, Medallion architecture and metadata-driven architecture; awareness of cost management and performance

Soft Skills

- Strong problem-solving and analytical thinking

- Ability to work in cross-functional teams and to influence without authority

- Ownership mindset and attention to detail

- Good communication skills with both technical and business stakeholders: leads with the recommendation, quantifies, and is transparent about uncertainty

? What we’re looking for in you:

- Led an Airflow implementation or migration (ideally from ADF), or run it at scale.

- Built metadata-driven orchestration or reusable ETL frameworks.

- Built developer tooling or LLM-agent workflows (MCP, skills, automated review) for engineering teams.

- Applied FinOps practices on Azure and Databricks: cost attribution, budgets, forecasting and measurable savings

- Experience running Airflow on Kubernetes or operating containerized workloads on AKS

? Why you'll love working with us

At Capitole, we believe our people are our greatest strength. That's why we offer:

- €1,200/year for your professional development (trainings, events, certifications…)

- Monthly feedback sessions with your team to keep learning and growing

- Private health insurance fully covered by Capitole

- Payflow benefits (meal, transport, childcare vouchers)

- Wellhub subscription

- Discounts with top brands through Capitole Club

?Life at Capitole

- Team buildings every two months (our summer party and Christmas dinner are legendary!)

- Football, padel, and basketball teams sponsored by Capitole

- Tech communities where sharing knowledge is part of our DNA

- And above all… an AMAZING TEAM that you’ll love being a part of!

Want to learn more about us? capitole-consulting.com

See what our teams are saying about us: Glassdoor Reviews

? Ready to take the next step?

Send us your profile – we can’t wait to meet you!

The employee will adhere to information security policies:

- Will have access to confidential information related to Capitole and the project they are working on.

- Must comply with the security and internal policies of both the company and the client.

- Must sign an NDA (Non-Disclosure Agreement).

📌 Hibrido BCN - Senior Data Platform Engineer (Barcelona)
🏢 Capitole
📍 Barcelona