About The Role As a Remote Data Annotator, you will perform end-to-end data annotation and AI/ML evaluation work that powers training datasets for NLP, computer vision, and large language model evaluation. You will follow detailed annotation guidelines, document edge cases, and contribute to training data quality initiatives that improve model performance.
What You Will Do Create and validate labels for text and image data
Perform named entity recognition and taxonomy tagging
Run content safety labeling using policy-based classification
Conduct prompt evaluation and response grading for LLM behavior
Execute RLHF ranking and pairwise comparisons
Complete QA checks, audits, and improve inter-annotator agreement
Write clear rationales for complex or subjective judgments




Collaborate asynchronously with distributed teams and project leads
Required Qualifications Mid-to-senior experience in data annotation, data labeling, or QA evaluation
Strong written reasoning and ability to maintain guideline adherence
Familiarity with LLM concepts, RLHF-style ranking, and prompt evaluation
Comfort using labeling tools, spreadsheets, and structured workflows
Quality Standards Success is measured by audit scores, guideline adherence, low rework rates, strong inter-annotator agreement, consistent throughput, and clear rationales that improve training data quality.
This is a Remote, FULL_TIME role.
#J-18808-Ljbffr

📌 Remote Data Annotator Jobs Madrid (Zaragoza)
🏢 Rex.zone
📍 Zaragoza