LLM Language Evaluation Specialist – Spain

Project-based | Spain | Remote
We are looking for Catalan language specialists to participate in a long-term LLM Language Evaluation Project focused on evaluating the quality and accuracy of AI-generated content.
The project is expected to continue for 3+ months, with selected candidates first completing a trial evaluation before joining the production stage.

? What You Will Do
- Evaluate AI/LLM-generated content in Catalan according to detailed project guidelines.
- Assess whether the content is accurate, natural, and linguistically appropriate.
- Identify and classify language-related issues, including:
- Mistranslation or inaccurate translation
- Missing or omitted content
- Incorrect terminology or word usage
- Spelling, grammar, punctuation, and spacing errors
- Word- or phrase-level issues
- Other issues affecting the quality and readability of the content
- Identify abnormal cases, such as non-Slovak content, language identification errors, duplicate content, or content requiring rejection.




- Apply evaluation criteria consistently and provide structured evaluation results.
- Maintain high accuracy and attention to detail throughout the project.

? Requirements
- High proficiency in Catalan.
- Previous experience with Large Language Model (LLM), AI evaluation, AI training, data annotation, localization, translation, or related projects is required.
- Strong understanding of Slovak grammar, vocabulary, terminology, and natural language usage.
- Strong ability to identify subtle language errors and distinguish between acceptable and incorrect expressions.
- Ability to understand and follow detailed evaluation guidelines.
- Strong attention to detail and execution ability.
- Comfortable working with structured digital documents and online tools.

Relevant professional or industry experience is a plus, but not mandatory.

? Project Information
- Project: LLM Language Evaluation
- Target Language: Catalan
- Duration: 3+

📌 LLM Language Evaluation Specialist (San Sebastián)
🏢 Thoth AI
📍 San Sebastián