ph3Responsibilities /h3ulliDesign and implement automated and manual QA strategies for GenAI systems /liliBuild regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety /liliValidate prompt logic, model behavior and evaluation metrics /liliExecute tests and debug pipelines /liliSupport deployment of GenAI products /liliLead system design reviews, advocating best practices for architecture and reliability of GenAI applications /liliCollaborate closely with data scientists and developers to embed testing and quality checkpoints /liliEstablish testing standards and workflows for continuous integration and deployment /liliConduct code reviews with a focus on design, testability, and maintainability /liliDocument system design decisions, test cases, and best practices /liliClose collaboration with domain experts and stakeholders to validate developed solutions /li /ulh3Requirements /h3ulliStrong software engineering background with QA ownership /liliFamiliarity with RAG Graph frameworks /lili3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems,



preferably including at least 1 year experience in GenAI systems /liliAdvanced Python skills /liliExperience working with LLM-based systems in real environments /liliHands-on experience with: LLM APIs, AWS Bedrock (Anthropic OSS), Azure Foundry (OpenAI), GCP (Gemini), Cloud platforms (AWS, Azure, GCP) /liliProven ability to implement test frameworks for GenAI applications /liliKnowledge of API testing, data validation, and data pipeline testing /liliExperience with version control (Git) /liliExperience with building and deploying AI/ML products through the full product lifecycle /liliGood understanding of scientific computing and ML frameworks /liliSolid understanding of safety, bias, and failure modes of GenAI systems /liliComfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued /liliProactive problem-solver who can anticipate needs and find solutions /li liComfortable working independently, but also a strong team player. /li /ul /p #J-18808-Ljbffr

📌 Senior Engineer – GenAI Quality Assurance (Madrid)
🏢 Jobtailor
📍 Madrid