Responsibilities- Design and implement automated and manual QA strategies for GenAI systems- Build regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety- Validate prompt logic, model behavior and evaluation metrics- Execute tests and debug pipelines- Support deployment of GenAI products- Lead system design reviews, advocating best practices for architecture and reliability of GenAI applications- Collaborate closely with data scientists and developers to embed testing and quality checkpoints- Establish testing standards and workflows for continuous integration and deployment- Conduct code reviews with a focus on design, testability, and maintainability- Document system design decisions, test cases, and best practices- Close collaboration with domain experts and stakeholders to validate developed solutionsRequirements- Strong software engineering background with QA ownership- Familiarity with RAG & Graph frameworks- 3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems,



preferably including at least 1 year experience in GenAI systems- Advanced Python skills- Experience working with LLM-based systems in real environments- Hands-on experience with: LLM APIs, AWS Bedrock (Anthropic & OSS), Azure Foundry (OpenAI), GCP (Gemini), Cloud platforms (AWS, Azure, GCP)- Proven ability to implement test frameworks for GenAI applications- Knowledge of API testing, data validation, and data pipeline testing- Experience with version control (Git)- Experience with building and deploying AI/ML products through the full product lifecycle- Good understanding of scientific computing and ML frameworks- Solid understanding of safety, bias, and failure modes of GenAI systems- Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued- Proactive problem-solver who can anticipate needs and find solutions- Comfortable working independently, but also a strong team player.#J-18808-Ljbffr

📌 Senior Engineer – Genai Quality Assurance (Madrid)
🏢 Jobtailor
📍 Madrid