Overview In this role you will own and evolve Zurich's Databricks platform to support the General Finance Data Hub, ensuring scalable, secure, and cost-conscious data solutions. You will build and maintain ingestion and processing pipelines, govern data with Unity Catalog, and implement CI/CD and IaC practices in a collaborative, tech-driven environment. You'll work with cross-functional teams to deliver a robust enterprise data platform that accelerates Zurich's data strategy and operational efficiency.

This is a chance to shape a world-class data foundation within a global insurer.

Compensaciones / Beneficios flexible working model abroad up to 25 days yearly home office allowance wide trainings including language classes health insurance life and retirement plans

Responsabilidades

Own and evolve the Databricks platform underpinning the Global Finance Data Hub with focus on scalability, reliability, security, and cost efficiency

Design, build, and maintain data ingestion frameworks for batch, streaming, API, and CDC across multiple sources

Develop and manage Delta Live Tables pipelines with data quality, schema enforcement, and CDC

Administer and optimize Delta Lake environments (performance, schema evolution, time travel, VACUUM, OPTIMIZE)

Implement Unity Catalog governance (access controls, lineage, classification, namespaces)

Build, deploy, and support Databricks Workflows for complex pipelines and event-driven processing

Establish CI/CD pipelines using Git and Databricks Asset Bundles across dev/test/prod





Collaborate with cloud/infrastructure teams on IaC using CloudFormation or CDK

Manage authentication/authorization (IAM roles, profiles, service principals, secrets)

Monitor platform health and costs with Databricks system tables, CloudWatch, Grafana

Define cluster policies, compute strategies, and DBU governance for optimization

Ensure security (PrivateLink, VPC endpoints, row/column security)

Partner with data engineering, finance, architecture, and product teams to deliver scalable data platform

Drive best practices, automation, and continuous improvement across the data ecosystem

Requisitos principales 5-8 years in Data Engineering/Platform Engineering/Cloud Data Architecture

Minimum 3 years owning and managing Databricks platforms in enterprise environments

Strong expertise in Databricks services (Unity Catalog, Delta Lake, Delta Live Tables, Workflows, Asset Bundles)

Experience designing/supporting batch, streaming, API, and CDC data ingestion frameworks

Advanced PySpark, Python, and Databricks SQL for scalable pipelines

DevOps practices including Git, CI/CD, automated testing, and IaC (CloudFormation or CDK)

AWS experience (S3, IAM, networking, security, secrets)

Experience with large-scale data platforms in regulated industries; financial services desirable

Preferred: Databricks/AWS certifications; Airflow, Kafka/MSK; data governance frameworks collaboration effective communication problem-solving

Databricks Unity Catalog

Delta Lake

Delta Live Tables

📌 Databricks Platform Engineer (Barcelona)
🏢 Zurich Insurance Group
📍 Barcelona