15 oct
|
TechShack
|
Castilla-La Mancha
15 oct
TechShack
Castilla-La Mancha
Databricks Data Engineer | 12 Month+ B2B Contract | Fully Remote | €500 per day I'm partnering with a general technology consultancy on a large data transformation programme for a well-established organisation in the insurance sector. They need an experienced, hands‑on Databricks Data Engineer to join the delivery team on a long‑term contract. The Role The end client is moving away from legacy data warehousing and scattered on‑premise systems to a modern, unified Lakehouse platform built on Databricks. You will sit at the centre of this transformation. You'll design and build scalable data pipelines, migrate legacy workloads, and help set the engineering standards the wider platform will be built on. You’ll work closely with architects, analysts and business stakeholders across the underwriting, claims, policy and finance data domains. This is a hands‑on engineering role. You should be comfortable writing production‑grade PySpark and SQL every day. The Project * Migrating legacy data warehouse and ETL workloads (SQL Server, SSIS, Informatica or similar)
onto Databricks * Designing and building a Medallion (Bronze, Silver, Gold) Lakehouse architecture on Delta Lake * Rolling out Unity Catalog for central governance, lineage and access control * Delivering curated, trusted data products for actuarial, risk, claims and regulatory reporting * Introducing CI/CD, automated testing and infrastructure as code across the platform * Enabling downstream BI, analytics and future AI/ML use cases Key Responsibilities * Design, build and maintain scalable batch and streaming pipelines on Databricks using PySpark, Spark SQL and Delta Lake * Migrate legacy ETL processes and data models onto the Lakehouse, keeping data accurate and reconciled throughout * Implement Medallion architecture patterns and build reusable ingestion and transformation frameworks * Orchestrate workflows using Databricks Workflows, Delta Live Tables, Azure Data Factory or Airflow * Work within Unity Catalog to
📌 Databricks Data Engineer (Castilla-La Mancha)
🏢 TechShack
📍 Castilla-La Mancha