Responsibilities * Design and implement automated and manual QA strategies for GenAI systems * Build regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety * Validate prompt logic, model behavior and evaluation metrics * Execute tests and debug pipelines * Support deployment of GenAI products * Lead system design reviews, advocating best practices for architecture and reliability of GenAI applications * Collaborate closely with data scientists and developers to embed testing and quality checkpoints * Establish testing standards and workflows for continuous integration and deployment * Conduct code reviews with a focus on design, testability, and maintainability * Document system design decisions, test cases, and best practices * Close collaboration with domain experts and stakeholders to validate developed solutions Requirements * Strong software engineering background with QA ownership * Familiarity with RAG & Graph frameworks * 3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems,



preferably including at least 1 year experience in GenAI systems * Advanced Python skills * Experience working with LLM-based systems in real environments * Hands-on experience with: LLM APIs, AWS Bedrock (Anthropic & OSS), Azure Foundry (OpenAI), GCP (Gemini), Cloud platforms (AWS, Azure, GCP) * Proven ability to implement test frameworks for GenAI applications * Knowledge of API testing, data validation, and data pipeline testing * Experience with version control (Git) * Experience with building and deploying AI/ML products through the full product lifecycle * Good understanding of scientific computing and ML frameworks * Solid understanding of safety, bias, and failure modes of GenAI systems * Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued * Proactive problem-solver who can anticipate needs and find solutions * Comfortable working independently, but also a strong team player. #J-18808-Ljbffr

📌 Senior Engineer – GenAI Quality Assurance (Madrid)
🏢 Jobtailor
📍 Madrid