Senior Azure Databricks Data Engineer - Banking Client

Location: Brussels, Belgium – Hybrid

Rate: Flexible

Onsite: 8 days per month in Brussels

Remote: Remaining days can be worked remotely

Engagement: Umbrella preferred; Belgian company/BV can also be considered

Duration: 6 months

The Opportunity

We are looking for an experienced Senior Azure Databricks Data Engineer to join a large-scale enterprise data transformation programme within a complex financial services environment.

This is a hands-on engineering position for someone who enjoys designing, developing and optimising production-grade data solutions rather than operating purely at architecture or management level.

You will work within a modern Microsoft Azure and Databricks ecosystem, building scalable data pipelines, Lakehouse solutions and reusable engineering components that support analytics, applications and emerging AI/ML use cases.

The role combines Data Engineering, Databricks/Spark development and software engineering, so we are particularly interested in engineers with strong coding skills and experience taking data solutions from design through to production.

What You'll Be Doing:

Azure Databricks & Data Engineering

- Design, build, test and maintain scalable data engineering solutions using Microsoft Azure and Azure Databricks.
- Develop production-grade ETL/ELT pipelines using Python, PySpark, Scala and SQL.
- Build data-processing applications covering ingestion, transformation, enrichment, validation and serving.
- Develop Delta Lake / Lakehouse solutions using modern data engineering patterns.
- Design and implement Bronze, Silver and Gold / Medallion architectures where appropriate.
- Build reliable batch and streaming data-processing workflows.
- Develop curated datasets and transformation layers supporting analytics, reporting, applications and AI/ML use cases.

Databricks & Apache Spark

- Develop and maintain Databricks notebooks, jobs and workflows.
- Build and optimise Apache Spark / PySpark workloads operating across large datasets.
- Improve Spark performance through appropriate partitioning, caching, cluster configuration and query optimisation.
- Implement schema evolution, incremental processing and robust data-quality controls.
- Develop reusable Spark/Python components, libraries and engineering frameworks.
- Apply appropriate Delta Lake optimisation and data-management techniques.
- Troubleshoot and optimise production data pipelines for performance, scalability, reliability and cost.

Azure Data Platform

Work across a modern Azure data ecosystem including:

- Azure Databricks
- Apache Spark / PySpark
- Delta Lake
- Azure Data Lake Storage (ADLS)




- Azure Data Factory
- Azure Synapse
- Azure Event Hubs / streaming patterns
- Azure Key Vault
- Azure DevOps
- Azure monitoring and logging capabilities

You will work closely with Cloud, Architecture and Platform teams to ensure solutions are secure, scalable, observable and aligned with enterprise standards.

Software Engineering & Application Development

This role goes beyond traditional ETL development.

You will apply strong software engineering practices to data applications, including:

- Modular and reusable development
- Clean, maintainable code
- Automated testing and validation
- Error handling and logging
- Code reviews
- Git/version control
- Technical documentation
- Reusable libraries and frameworks

You will be expected to contribute to the overall quality of the engineering environment rather than simply delivering individual pipelines.

DevOps & CI/CD

- Build and maintain CI/CD processes for data applications.
- Deploy Databricks and data-engineering code across development, test and production environments.
- Work with Azure DevOps and YAML pipelines.
- Collaborate with DevOps and Cloud teams on environment configuration and deployment.
- Apply release-management and environment-promotion best practices.
- Work with Terraform / Infrastructure as Code where required.

Terraform expertise is beneficial, but this is primarily a Data Engineering and application-development role rather than an Infrastructure Engineering position.

Data Quality, Security & Governance

- Build data-quality controls and validation into engineering pipelines.
- Implement appropriate logging, monitoring and operational alerting.
- Work with enterprise security and access-management standards.
- Apply secure coding and cloud data-engineering practices.
- Support metadata, lineage and governance requirements.
- Consider performance and cloud cost when designing and developing solutions.

Technical Leadership

As a senior member of the engineering team, you will also:

- Work closely with Data, Architecture, AI, Cloud, DevOps and application teams.
- Translate complex business and analytical requirements into practical engineering solutions.
- Contribute to technical design and engineering standards.
- Conduct code reviews and provide technical guidance.




- Support and mentor less experienced engineers.
- Help establish reusable development patterns and engineering best practices.

What We're Looking For

You should have at least 5 years of hands-on Data Engineering / Data Application Development experience, ideally within large enterprise environments.

Core Technical Skills

Strong hands-on experience with:

- Azure Databricks
- Python
- PySpark
- Apache Spark
- Scala
- SQL
- ETL / ELT pipeline development
- Delta Lake / Lakehouse architecture
- Azure Data Factory
- Azure Data Lake Storage
- Azure DevOps / CI/CD

We are particularly interested in candidates who can demonstrate that they have personally designed and developed production Databricks/PySpark solutions, rather than only managing teams or defining architecture.

Highly Desirable

Experience with any of the following would be advantageous:

- Spark performance optimisation
- Databricks Auto Loader
- Delta Live Tables
- Unity Catalog
- Medallion / Bronze-Silver-Gold architecture
- Streaming data pipelines
- Azure Event Hubs
- Kafka
- Azure Synapse
- Azure Key Vault
- Azure DevOps YAML
- Terraform
- Data quality frameworks
- Metadata management and lineage
- MLflow / MLOps
- AI/ML data pipelines
- Reusable data-engineering frameworks or libraries
- Enterprise security and governance

Certifications

Relevant certifications are advantageous, particularly:

- Microsoft Azure Data Engineer
- Microsoft Azure Developer
- Microsoft Azure Solutions Architect
- Microsoft Azure DevOps Engineer
- Databricks Certified Data Engineer Associate
- Databricks Certified Data Engineer Professional
- Databricks Certified Developer for Apache Spark

The Profile That Will Stand Out

The strongest candidate will be someone who can say:

"I personally build production Azure Databricks solutions. I write Python/PySpark code, build and optimise Spark pipelines, work with Delta Lake and ADF, automate deployments through CI/CD, troubleshoot production issues and understand how to engineer scalable data solutions rather than simply design them."

This is not primarily a BI, reporting or high-level Data Architecture position. We are looking for a genuinely hands-on senior engineer who is comfortable getting into the code.

Working Model

The position offers a highly versátil hybrid model:

8 days per month onsite in Brussels, with the remainder of the month worked remotely.

The preferred engagement model is via an Umbrella solution. Consultants operating through their own Belgian company/BV may also be considered.

Candidates must be comfortable committing to the 8 days per month onsite in Brussels.

📌 Senior azure databricks data engineer - banking client (España)
🏢 Salt
📍 España