AI Evaluation SpecialistRole Type: ContractorLocation: Remote — US, Canada, UK, Ireland, Australia, New ZealandWe are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project.You'll evaluate AI-generated responses using defined quality standards, identify issues, and provide clear feedback to help improve AI systems. No prior formal AI experience is required.Scope of Work- Evaluate AI-generated outputs against detailed rubrics for accuracy, relevance, quality, and adherence to guidelines.- Apply consistent and objective judgment across large volumes of examples.- Identify reasoning gaps, tool-use issues, logical errors, and inconsistencies.- Provide concise,



actionable feedback highlighting strengths and areas for improvement.- Participate in discussions around rubric interpretation and evolving quality standards.- Contribute to process improvements and evaluation best practices.- Maintain accurate documentation of evaluations and recommendations.Preferred Qualifications- Experience in grading, quality assurance, editorial review, assessment, annotation, or similar analytical work.- Advanced, daily use of AI assistants such as ChatGPT, Claude, or similar tools.- Strong ability to analyze complex information and communicate findings clearly in writing.- Experience with process improvement, rubric development, or quality assessment.- Strong critical-thinking skills with a focus on consistency, accuracy, and fairness.- Comfortable working independently on high-volume evaluation tasks.- Strong attention to detail and ability to identify subtle quality issues.- Collaborative approach to resolving ambiguous cases and improving evaluation criteria.

📌 Ai Evaluation Specialist (Barcelona)
🏢 YO AI Labs
📍 Barcelona