to join a central Data Delivery team building the foundation of a integral
Data Mesh platform based on a Lakehouse architecture
.
You will work on the engineering layer responsible for ingesting, processing and provisioning source-aligned data products into central data catalogs, enabling teams across the organisation to consume trusted data for analytics, reporting, machine learning and other data-driven applications.
A key part of the platform is enabling scalable data consumption through
zero-copy data sharing
, while maintaining strong governance, security and data quality standards.
What You'll Be Working On
Design and build scalable
batch and streaming data pipelines
.
Develop ingestion solutions for source-aligned data products.
Work with
Apache Spark and Databricks
within a modern Lakehouse environment.
Implement data ingestion strategies including
Full Loads, Delta Loads and Change Data Capture (CDC)
.
Build and maintain streaming pipelines using technologies such as
Kafka, Flink or Confluent
.
Manage datasets stored across
Google Cloud Storage and Azure Blob Storage
.
Enable secure
zero-copy data sharing
through technologies such as Databricks Unity Catalog and BigQuery.
Implement data governance and access-control models including
RBAC and attribute-based access control
.
Design solutions capable of handling complex
schema evolution
, including backward and forward compatibility.
Tech Environment
Data & Processing: