Job title: Senior Data Engineer
Country/Corporate/Region:
Regional
Function:
Digital Transformation & AI
Department:
GDS
Directly reporting to:
Head of Data & AI in EDC
Other reporting lines:
Job Level:
Senior data engineer
SUMMARY OF THE JOB
We are seeking a seasoned Senior Data Engineer to design, build, and optimize our next-generation data platform. You will be responsible for architecting scalable data pipelines, managing large-scale distributed systems, and ensuring our data infrastructure in AWS and Databricks is robust and efficient. The adecuado candidate is a Spark expert with a deep understanding of the AWS ecosystem and a passion for automation.
MAIN ACTIVITIES / RESPONSIBILITIES
-
Pipeline Architecture: Design and implement complex batch and streaming ETL/ELT pipelines using Python, SQL, and Spark to process massive datasets.
-
Cloud Infrastructure: Leverage AWS Data Analytics services to build scalable, secure, and cost-effective data solutions.
-
Orchestration & DevOps: Manage and automate data workflows using Airflow,
while utilizing Docker and ECS for containerized application deployment.
-
System Optimization: Monitor and tune the performance of distributed systems (Spark Cluster) to ensure high availability and low latency.
-
Infrastructure as Code: Utilize AWS CloudFormation or Terraform to manage data infrastructure, ensuring repeatable and version-controlled environments.
-
Cost Optimization: Monitor and optimize AWS spend by selecting appropriate instance types (Spot vs. On-Demand) and refining data storage strategies.
-
Security & Compliance: Implement IAM roles, bucket policies, and encryption (KMS) to ensure data is secure at rest and in transit.
-
Collaboration: Work within an Agile framework to deliver iterative value, collaborating closely with Data Scientists and Stakeholders to translate business needs into technical reality.
JOB DIMENSIONS
List of direct reports:
-
Up to 2 Direct Reports, and around 15 externals
Ke
📌 Senior Data Engineer (España)
🏢 Holcim
📍 España