Responsibilities Design and implement automated and manual QA strategies for Gen AI systems Build regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety Validate prompt logic, model behavior and evaluation metrics Execute tests and debug pipelines Support deployment of Gen AI products Lead system design reviews, advocating best practices for architecture and reliability of Gen AI applications Collaborate closely with data scientists and developers to embed testing and quality checkpoints Establish testing standards and workflows for continuous integration and deployment Conduct code reviews with a focus on design, testability, and maintainability Document system design decisions, test cases, and best practices Close collaboration with domain experts and stakeholders to validate developed solutions Requirements Strong software engineering background with QA ownership Familiarity with RAG & Graph frameworks 3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems,



preferably including at least 1 year experience in Gen AI systems Advanced Python skills Experience working with LLM-based systems in real environments Hands-on experience with: LLM APIs, AWS Bedrock (Anthropic & OSS), Azure Foundry (Open AI), GCP (Gemini), Cloud platforms (AWS, Azure, GCP) Proven ability to implement test frameworks for Gen AI applications Knowledge of API testing, data validation, and data pipeline testing Experience with version control (Git) Experience with building and deploying AI/ML products through the full product lifecycle Good understanding of scientific computing and ML frameworks Solid understanding of safety, bias, and failure modes of Gen AI systems Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued Proactive problem-solver who can anticipate needs and find solutions Comfortable working independently, but also a strong team player. #J-18808-Ljbffr

📌 Senior Engineer – Genai Quality Assurance (Madrid)
🏢 Jobtailor
📍 Madrid