Senior AI Test Automation Engineer
We are looking for the very Top Talent and we would be delighted if you were to join our team
More in detail, UST is a multinational company based in North America, certified as a Top Employer and Great Place to Work, with over 35,000 employees worldwide and presence in more than 35 countries. We are leaders in digital technology services, delivering large-scale technological solutions to top-tier companies.
What we look for?
We are looking for a Senior AI Test Automation Engineer to join the ML AI ART, owning the end-to-end test automation strategy for an advanced AI/MLOps platform. This platform powers business-critical AI capabilities such as LLM-based assistants, RAG systems, agentic workflows, and ML pipelines.
Main tasks and accountabilities will be:
- Define and evolve the test automation architecture aligned with the Bank s Test Strategy
- Design scalable, reusable automation frameworks and patterns across teams
- Analyse requirements and ensure testability during Agile ceremonies (PI Planning, Backlog Refinement, etc.)
- Design and automate test cases (functional non-functional) based on risk and coverage
- Implement automation using Python frameworks (Playwright, pytest, Behave/BDD)
- Develop AI/ML-specific testing approaches (LLM evaluation, non-deterministic behaviour, RAG workflows)
- Build and maintain API contract tests and microservices validation
- Integrate and run automated tests in CI/CD pipelines (GitLab)
- Use Docker/Kubernetes for reproducible test environments
- Manage test execution across environments (DEV, INT, UAT, pre-PROD)
- Monitor test health, handle flaky tests, and ensure reliable reporting
- Report defects in Jira with proper traceability and evidence
- Produce audit-ready test artefacts and ensure compliance with governance standards
- Collaborate with cross-functional teams (POs, Engineers, MLOps, Data Engineers)
- Act as Test Automation Ambassador within the ART
- Ensure end-to-end traceabi
📌 Senior AI Test Automation Engineer, hibrido (España)
🏢 Ust
📍 España