12 nov
|
Farmatalento
|
Madrid
12 nov
Farmatalento
Madrid
We are looking for a Lead Data & Platform Engineer to join the Health Data Platform and Innovation Unit of a Spanish healthcare Consulting, Research and Innovation company, founded in 2022 and currently undergoing significant growth.
MISSION
Technical ownership of the Data Platform: Architecture, Data Model, Source Integration, Continuous Ingestion, Quality, Performance and Operations.
This is not a maintenance role, nor one focused on implementing decisions made by others: the person in this position will define the architecture, document it and take ownership of its long-term operation and evolution.
The role has four main areas of responsibility:
1. Consolidate and scale the Clinical Data Platform. Redesign the Data Model, Performance and Traceability to support growth in data volume, data sources and the number of simultaneous projects.
2. Integrate new Healthcare Data Sources. Incorporate new Clinical, Hospital and Administrative Data Sources into the common model, addressing the heterogeneity of formats, terminologies and identifiers inherent to the healthcare sector.
3. Event-Driven Architecture and Interoperability. Design and operate Continuous Ingestion and Processing of Healthcare Data using Event-Driven Architectures, with HL7 FHIR Interoperability and under strict data security and sovereignty requirements. This is a core part of the role, not an additional responsibility.
4. Technical Standards and Team. Define the Unit's Engineering practices, document the Architecture and technical Decisions, and participate in building and developing the team as it grows. You will work closely with Data Analysts, Data Scientists,
Biostatisticians and the Scientific and Clinical Teams to ensure that data is available, validated, traceable and ready for use across projects. You will report to the Head of the Unit.
TECHNOLOGY STACK
- On-premise PostgreSQL (proprietary clinical data model): tuning, partitioning, RLS and replication.
- Python for data processing, integration and automation.
- Apache Kafka / Confluent Platform: event ingestion and processing.
- Change Data Capture from healthcare systems (Debezium, Kafka Connect).
- Distributed event processing: Kafka Streams, Apache Flink or ksqlDB.
- Schema governance and data contracts: Schema Registry, Avro/Protobuf, versioning and compatibility.
- Healthcare interoperability: HL7 FHIR R4, HL7v2, CDA. Clinical terminologies: SNOMED CT, ICD-10, ATC, LOINC.
- Common Data Models: OMOP CDM and other applicable European standards.
- Orchestration and version-controlled transformation: Airflow or Dagster, dbt.
- Kubernetes, Docker, CI/CD, and on-premise and hybrid-cloud deployments.
- LLM and agent pipelines operating on structured clinical data.
Absolute expertise across the entire technology stack is not expected. However, Event-Driven Architectures and production operations should be familiar territory, together with sufficient technical judgement to make informed decisions across the remaining areas.
KEY RESPONSIBILITIES
Event-Driven Architecture
- Define the architecture for event ingestion and processing: cluster topology, topic design and partitioning, delivery guarantees, idempotency, retention, replication and disaster recovery.
- Implement Change Data Capture and in-transit validation for healthcare data.
- Design the interoperability layer: transformation into FHIR R4 resources, profiles and Implementation Guides, and their representation as data contracts.
- Translate clinical and regulatory requirements into verifiable architectural decisions covering latency, traceability, auditability and data quality in transit.
- Work with clinical teams to translate clinical validation and plausibility rules into executable logic across data flows.
Platform and Data Model
- Redesign and optimise the PostgreSQL relational model: partitioning, indexing, study/user-level RLS, tuning and replication.
- Design, manage and optimise databases for clinical and Real-World Data, ensuring quality, integrity, traceability and security throughout the entire data flow.
- Define the data architecture required to support the company's growth in terms of sources, volume and clients.
Integration of New Data Sources
- Build extraction pipelines from EHR systems, including structured and unstructured data, clinical records and administrative data.
- Incorporate new sources into the common model: source analysis, mapping, patient identifier resolution and data-load quality control.
- Implement and maintain ETL/ELT processes for integrating, transforming and loading data from multiple sources.
- Curate and normalise data into CDMs (Common Data Models). Map data to clinical
📌 Lead Data (Madrid)
🏢 Farmatalento
📍 Madrid