Join a project shaping the future of sustainable energy through cutting‑edge data engineering and scientific innovation. We are looking for a Data Engineer to help build scalable cloud-native data platforms for a deep‑tech Customer working on one of the most ambitious fusion energy initiatives in the world.

You will work with modern distributed data technologies, including Databricks, Apache Spark, AWS, and Azure, while solving complex engineering challenges in a research‑intensive environment.

We at Sigma Software value engineering excellence, long‑term partnerships, and continuous learning. This is an opportunity to contribute to breakthrough scientific initiatives while working with modern technologies and highly skilled general teams.

PROJECT The project focuses on building and evolving a modern cloud‑based data platform supporting scientific research, analytics, and operational workflows in the fusion energy domain. The team develops scalable distributed data processing solutions while ensuring high standards of data quality, lineage, governance, reproducibility, and operational reliability.

You will work closely with scientists and engineers to support experimentation, analytics, and machine learning workflows in a highly innovative R&D; environment

.Job Descriptio

Design, develop, and maintain scalable and reliable data platforms and pipelines for analytics and operational use case

Build distributed data processing workflows using Apache Spark, pySpark, and Databric

Develop and evolve cloud-native data solutions leveraging Azure and AWS services





Model and optimize data structures for analytics, reporting, and downstream consumption

Integrate data from multiple heterogeneous sources while ensuring data quality, consistency, and traceability

Implement data quality controls, schema validation, lineage tracking, and monitoring processes

Optimize performance, scalability, reliability, and cost‑efficiency of data pipelines and infrastructure

Collaborate closely with scientists, analysts, and engineering teams to translate business and research requirements into technical solutions

Contribute to CI/CD processes, infrastructure automation, and engineering best practices

Research and introduce new technologies and approaches within the data engineering ecosystem

ecosystemQual

3+ years of commercial experience in Data Engineering

2+ years of hands‑on production experience with Databricks

Strong expertise with Delta Lake, including medallion architecture and table optimization

Advanced proficiency in Python, pySpark, and SQL

Deep practical knowledge of Databricks components including Unity Catalog, Lakeflow Jobs / Workflows, and job optimization

Hands‑on experience with AWS services including S3, IAM,



KMS, and at least one of AWS Batch, ECR, EventBridge, Lambda, or API Gateway

Experience designing event‑driven and asynchronous data pipelines

Strong knowledge of Infrastructure as Code and CI/CD practices

Practical experience with Terraform and Databricks Asset Bundles

Experience working with Git workflows, pull request reviews, and automated testing

Understanding of secure authentication approaches such as OIDC and service principals

Practical experience with MLflow, experiment tracking, model registry, and reproducibility practices

Strong understanding of data quality, lineage, schema contracts, and provenance concepts

Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks DQXExperience monitoring and optimizing data platform reliability and cloud costs

Upper-Intermediate or higher English level

WILL BE A PLUS:

Experience packaging workloads using Docker and running batch workloads in AWS

Familiarity with ECR and containerized data processing solutions

Experience with experiment optimization libraries such as scikit‑optimize, BoTorch, Ax, or scipy

Understanding of ML feature engineering and leakage prevention techniques

Experience working in regulated or high‑security environments

Knowledge of data governance, compliance, and data residency requirements

Experience disabling or restricting AI‑assisted tooling/ features in secure environments

#J-18808-Ljbffr

📌 Data Engineer (Madrid)
🏢 Sigma Software
📍 Madrid