Overview

As AI QA Engineer in the AI for Coding Team, you will ensure reliability of both human and AI-generated code while shaping test strategies for AI-assisted development. You collaborate with developers and ML engineers to validate AI performance and maintain quality gates across releases. You design regression tests, oversee AI evaluation pipelines, and curate benchmarks to drive model improvement. This role combines traditional QA rigor with AI-focused evaluation in a fast-paced, mission-driven tech hub.

Compensaciones / Ventajas

- Hybrid remote-office workplace
- Flexible schedule
- Private Health insurance
- Restaurant Ticket for 170/month
- 24 Holiday days per year
- 1.000 Referral reward

Responsabilidades

- Define and execute test plans for human-written and AI-generated code
- Design regression testing for AI-assisted development workflows
- Collaborate with ML engineers to evaluate precision, recall, and correctness metrics




- Monitor automated test pipelines with AI evaluation steps
- Curate benchmark datasets for fine-tuning (SFT/DPO) based on QA insights
- Define and maintain quality thresholds for PR bot output
- Promote a data-driven QA culture focused on continuous AI model improvement

Requisitos principales

- 3+ years of experience as a QA Engineer testing web or mobile applications
- Solid understanding of software testing principles and automation frameworks (Cypress, Playwright)
- Experience with CI/CD pipelines and test integration tools
- Ability to analyze metrics and interpret AI outputs
- Hands-on experience with API testing tools (Postman, Charles Proxy)
- Interest in LLM evaluation and model monitoring workflows

- collaboration with cross-functional teams
- data-driven mindset
- proactive communication
- Cypress
- Playwright
- CI/CD tooling

📌 AI QA Engineer - barcelona
🏢 Melza Consulting
📍 Barcelona