Overview

In this role you will lead the evaluation and responsible deployment of clinical AI at Astra Zeneca, shaping how AI models are tested, validated, and scaled in clinical development. You will build evidence systems and release criteria that satisfy safety, regulatory, and operational needs, working with clinical, engineering, IT, and data teams. You will design rigorous experiments, performance monitoring, and governance to ensure trustworthy AI in patient-facing workflows. This is a hands-on leadership role that emphasizes impact, collaboration, and clear communication of risk and value.

Responsabilidades

- Define and implement the evaluation strategy for clinical AI models and agents across development, release, monitoring, and change control
- Build evaluation harnesses, curated test sets, simulation environments, automated regression suites, and analysis pipelines in Python and related technologies
- Translate expert judgments into task definitions, scoring rubrics, error taxonomies, and clinically meaningful acceptance thresholds
- Evaluate complete workflows—not only model outputs—considering retrieval quality, tool selection, orchestration, source fidelity, abstention, human hand-offs, latency, and downstream impact
- Design approaches for noisy, sparse, delayed, or expert-dependent ground truth, including adjudication, inter-rater agreement, challenge sets, prospective studies, and post-deployment surveillance
- Lead failure analysis and red-teaming for hallucination, unsupported claims, automation bias, data leakage, prompt injection, subgroup performance, and unsafe workflow behaviour
- Establish traceability from intended use through release decisions, including how model, prompt, tool, data, and workflow changes are assessed and revalidated
- Collaborate with Quality, Regulatory, Clinical Operations, Privacy, Security, and R&D; IT to align evidence with GCP, GxP, data-integrity, validation,



and inspection-readiness
- Advise teams on progression decisions from prototype to regulated use and broader deployment
- Create reusable evaluation components and standards for adoption across agentic clinical workflows
- Communicate findings and residual risks to clinical, quality, regulatory, and executive audiences; make uncertainty visible
- Mentor scientists and engineers in rigorous experimentation, reproducibility, and responsible clinical AI development

Requisitos principales

- PhD in a quantitative discipline or equivalent with strong computational track record OR MD with strong computational experience
- 4 to 7 years post-PhD experience evaluating, validating, or deploying AI/ML in healthcare or safety-critical contexts
- Hands-on coding in Python and SQL; building automated evaluation pipelines and large-scale data analysis in version-controlled environments
- Strong experimental design, statistical inference, uncertainty, error analysis, and measurement reliability
- Experience evaluating LLMs or agentic systems with retrieval, tool use, multi-step workflows, robustness, and human-in-the-loop performance
- Ability to design evaluation approaches in noisy or delayed settings with expert opinions and uncertain ground truth
- Audit-ready evidence production under GCP and GxP including model-risk management and inspection readiness
- Ability to work with clinical stakeholders to define intended use, failure modes, thresholds, and appropriate human oversight
- Deep understanding of trustworthy AI and lifecycle governance and translating them into concrete release/monitoring criteria
- Excellent communication and cross-functional influence to support evidence-based decisions
- Experience leading cross-functional technical work and producing go/no-go recommendations under ambiguity

- excellent communication
- cross-functional collaboration
- judgment under ambiguity
- Python
- SQL
- evaluation pipelines

📌 Associate Director, Clinical AI Evaluation and Responsible Deployment (Barcelona)
🏢 AstraZeneca
📍 Barcelona