- Keep production Databricks pipelines running, including ingestion, transformation, and delivery to downstream consumers
- Diagnose and resolve pipeline failures and data quality issues
- Reverse-engineer and document existing transformation logic and business rules
- Migrate legacy tables from Hive Metastore to Unity Catalog
- Maintain Iceberg-enabled table sharing between Databricks and Snowflake
- Build and test dbt models, including incremental materializations and data tests
- Develop and maintain Airflow DAGs for orchestration
- Validate migrated pipelines against Databricks outputs
- Contribute to Snowflake modeling, performance, and cost decisions
- Work directly with client stakeholders on technical topics alongside the team lead

Requirements
- 3+ years operating production data pipelines
- PySpark and SQL — able to read, debug, and modify existing pipelines
- Delta Lake: MERGE/upsert patterns, table properties, OPTIMIZE, partitioning
- Databricks Workflows, cluster configuration, job troubleshooting
- Unity Catalog: catalogs, schemas, grants, lineage,



and the metastore model
- Snowflake warehouses, roles and grants, and general operating model
- Snowflake query performance and awareness of compute cost behavior
- dbt models, sources, tests, and incremental materializations
- dbt project structure and deployment workflow
- Airflow DAGs, operators, scheduling, and dependency management
- Airflow retries, backfills, and idempotent task design
- Strong SQL, including window functions, complex joins, and reading transformation logic
- Python for scripting, automation, and API integration
- Incremental loading patterns, idempotency, late-arriving data, and reprocessing
- AWS S3 and IAM basics
- Basic working knowledge of Redshift and its role in wider architecture
- Fluent English
- Self-directed and able to progress on an unfamiliar codebase without structured onboarding
- Able to explain production incidents to non-technical stakeholders and provide realisti

📌 Mid Data Engineer (Barcelona)
🏢 Jobtailor
📍 Barcelona