Overview

In this role you will design, test, and refine GenAI-powered workflows and pipelines within Innovative Medicine R&D.; You will validate and stress-test AI systems across the lifecycle, partnering with scientists and developers to embed quality checkpoints. Your work ensures reliable, safe, and scalable GenAI solutions that support research and pre/post-release activities. This position offers hybrid, Madrid or Barcelona-based work and the chance to shape AI-enabled healthcare innovations.

Responsabilidades

- Design and implement automated and manual QA strategies for GenAI systems
- Build regression tests, generate synthetic test data, create test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety
- Validate prompt logic, model behavior and evaluation metrics
- Execute tests and debug pipelines
- Support deployment of GenAI products
- Lead system design reviews, promoting architecture reliability for GenAI applications
- Collaborate with data scientists and developers to embed testing and quality checkpoints
- Establish testing standards and workflows for CI/CD
- Conduct code reviews focusing on design, testability and maintainability
- Document design decisions, test cases,



and best practices
- Collaborate with domain experts and stakeholders to validate solutions

Requisitos principales

- Strong software engineering background with QA ownership
- 3+ years as software engineer or QA in production systems (5+ preferred)
- Advanced Python skills
- Experience with LLM-based systems in real environments
- Hands-on with LLM APIs
- Experience with AWS Bedrock, Azure Foundry, GCP Gemini
- Cloud platforms knowledge (AWS, Azure, GCP)
- Proven ability to implement test frameworks for GenAI applications
- Knowledge of API testing, data validation, and data pipeline testing
- Version control experience (Git)
- Experience deploying AI/ML products through full lifecycle
- Good understanding of scientific computing and ML frameworks
- Understanding of safety, bias, and failure modes in GenAI
- Ability to work independently and as part of a team
- Nice-to-have: Jenkins, SonarQube, Prometheus, Grafana, Docker

- Proactive problem-solver
- Strong collaboration with cross-functional teams
- Adaptability in fast-moving environments
- LLMs
- Python
- API testing

📌 Sr Engineer – GenAI Quality Assurance - Johnson & Johnson (Barcelona)
🏢 Johnson & Johnson
📍 Barcelona