Capitole is growing fast, and we want you to be part of the journey!

We are looking for a

Senior Data Platform Engineer

with

+6 years of experience

to help drive the evolution of our

Azure-based Data Platform

and

Lakehouse architecture . This role is focused on the

platform itself . The professional will own and evolve the

foundations

that data product teams build on:

Apache Airflow orchestration (migrating from ADF) ,

Azure resource management

(Key Vault, identity and access, networking, storage),

Azure DevOps CI/CD and Infrastructure as Code ,

Databricks and Unity Catalog governance ,

observability

and

cost efficiency , while remaining

hands‑on

and close to the teams that use the platform.

Responsibilities: Orchestration Platform (Apache Airflow & ADF)

Own the

Apache Airflow

platform: environments, deployment, upgrades, configuration, connections, variables, pools and

DAG standards .

Work on the migration from

Azure Data Factory to Airflow

and the evolution of the

Metadata-Driven Orchestration Framework

(DAG generation, reusable operators, dependency and scheduling patterns).

Ensure Airflow is

reliable and observable : SLAs, alerting, retries, backfills, health monitoring and clear failure ownership.

Azure Resource Management

Manage Azure resources with

Infrastructure as Code (Terraform) : Databricks workspaces,

Data Lake Storage , Data Factory,

Key Vault , networking and supporting services.

Own

secrets and identity management : Key Vault, service principals and managed identities, secret scopes and rotation.

Own

access management : Azure RBAC, Entra ID groups, Unity Catalog grants and ownership, following

least privilege , plus access requests and periodic reviews.

Own

Azure DevOps : repositories,

CI/CD pipelines , branch policies and

multi‑environment promotion , including deployment validation and Terraform state.

Track

cost and governance

of Azure resources: tagging, budgets, right‑sizing and cost attribution (e.g. query tagging).

Hands‑on Data Platform evolution

Contribute to the implementation of the

Data Catalog

and

Data Quality Framework

and it’s

lineage

across the platform.

Drive

observability : pipelines that are observable, debuggable and alert‑driven, with clear ownership of failures.

Operate the

Databricks

platform:





Unity Catalog

structure, cluster and job policies,

Auto Loader

ingestion patterns, and performance and cost optimization of workloads.

Technical Leadership

Participate in

architecture reviews

and technical decision‑making, stating

trade offs

(performance, cost, reliability, complexity) with data.

Define and promote

engineering standards : naming conventions, patterns and guidelines, through code reviews and design reviews.

Collaborate with Architects, Product Owners and data engineers in

technical maturation

of data products.

Build and maintain

developer tooling and automation , including

AI‑assisted engineering

workflows (shared agent instructions, skills, automated PR review).

Support & Operations (Run Team Collaboration)

Provide

second‑line support

to the Run/Operations team for complex production topics.

Troubleshoot and resolve

platform incidents , failures and performance issues; lead

root cause analysis

and post‑mortems.

Ensure proper

handover, documentation and operational readiness

of platform changes.

Continuously improve

runbooks, monitoring and operational processes .

Requirements: Technical Skills

6+ years of experience in data engineering, with significant time on

platform or infrastructure

topics

Strong technical expertise with Python and SQL (advanced), including PySpark and

Delta Lake

semantics

Deep hands‑on experience with

Apache Airflow

in production: DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads

Hands‑on experience with

Azure resource management :

Key Vault , managed identities and service principals, secret management

Access management : Azure RBAC, Entra ID, Unity Catalog permissions

Azure DevOps

(repos, pipelines, service connections, environments) and CI/CD for data platforms

ADLS, networking and security basics, Azure Data Factory

Infrastructure as Code

(Terraform) and multi‑environment promotion





Hands‑on experience with

Databricks

and

Unity Catalog

(compute, jobs, governance)

Observability and monitoring , alerting and incident management

Solid understanding of Data Lakehouse architecture,

Medallion architecture

and

metadata‑driven architecture ; awareness of

cost management

and performance

Soft Skills

Strong problem‑solving and analytical thinking

Ability to work in cross‑functional teams and to

influence without authority

Ownership mindset and attention to detail

Good communication skills with both technical and business stakeholders: leads with the recommendation, quantifies, and is transparent about uncertainty

What we’re looking for in you:

Led an

Airflow

implementation or migration (ideally from ADF), or run it at scale.

Built

metadata‑driven orchestration

or reusable ETL frameworks.

Built developer tooling or

LLM-agent workflows

(MCP, skills, automated review) for engineering teams.

Applied FinOps practices on Azure and Databricks: cost attribution, budgets, forecasting and measurable savings

Experience running Airflow on Kubernetes or operating containerized workloads on AKS

Why you’ll love working with us At Capitole, we believe our people are our greatest strength. That's why we offer:

€1,200/year for your professional development (trainings, events, certifications…)

Monthly feedback sessions with your team to keep learning and growing

Private health insurance fully covered by Capitole

Payflow benefits (meal, transport, childcare vouchers)

Wellhub subscription

Discounts with top brands through Capitole Club

Life at Capitole

Team buildings every two months (our

summer party

and Christmas dinner are legendary!)

Football, padel, and basketball teams sponsored by Capitole

Tech communities where sharing knowledge is part of our DNA

And above all… an

AMAZING TEAM

that you’ll love being a part of!

Want to learn more about us?

capitole-consulting.com

See what our teams are saying about us: Glassdoor Reviews

Will have access to confidential information related to Capitole and the project they are working on.

Must comply with the security and internal policies of both the company and the client.

Must sign an NDA (Non-Disclosure Agreement).

#J-18808-Ljbffr

📌 Hibrido BCN - Senior Data Platform Engineer (Bellprat)
🏢 Capitole Consulting
📍 Bellprat