Position

LLM AI Quality Analyst (Spanish)

Type

Short-Term Contract

Compensation

$11 per hour

Location

Remote

Commitment

Part-time (30 40 hrs/week)

Role Responsibilities

Evaluate a personalization feature for Gemini

Design and execute multi-turn conversational prompts that require the AI to utilize personal information and experiences

Assess how effectively the model uses past conversations and activity to generate relevant and helpful responses

Evaluate model responses based on intent and appropriate personalization

Analyze responses for grounding issues, including flawed inferences or hallucinations

Assess integration quality to ensure personal data is incorporated naturally into responses

Perform side-by-side evaluations and stack-rank model responses based on helpfulness and naturalness

Write clear rationales referencing specific conversation turns

Extract and verify debug information to confirm correct use of summaries and data sources





Maintain data hygiene by deleting evaluation conversations after completion

Requirements

Experience in data annotation, AI quality evaluation, content moderation, or related roles

Strong Spanish proficiency (reading and writing)

Willingness to use a primary personal Google account and enable personal data sources for assessment

Strong analytical thinking and attention to detail

Experience with creative prompt engineering and personalization concepts

Ability to provide structured feedback and clear written explanations

Ability to work independently in a remote environment

Application Process

Apply/Easy Apply and check email for application form

Fill Google form

Assessment Link (After shortlisting to be completed within 24 hours)

📌 Generative AI Evaluator (Spanish) | $11/hr Remote (Madrid)
🏢 Crossing Hurdles
📍 Madrid