Remote Data Annotation Jobs Madrid (Full-Time, Remote)Join production-grade AI/ML training workflows by labeling and evaluating data used to improve model performance across NLP, computer vision, and content safety systems. You will follow strict rubrics and maintain strong annotation guidelines compliance to protect training data quality for LLM training pipelines.Key ResponsibilitiesExecute structured annotation tasks: classification, ranking, span labeling, and named entity recognition (NER)Perform LLM evaluation and prompt evaluation (helpfulness, factuality, groundedness, instruction-following)Contribute to RLHF workflows via preference judgments, rationale writing, and comparative response evaluationRun QA evaluation through self-checks, peer reviews, and audit responsesDocument edge cases, track error patterns,



and propose guideline clarifications to reduce varianceProjects and Task TypesMultilingual NER and text labelingContent safety labeling aligned to policy taxonomiesComputer vision annotation (bounding boxes, segmentation, image-text alignment) as neededCalibration sets and inter-review agreement improvement activitiesRequired QualificationsExperience with guideline-based labeling, evaluation frameworks, QA, data operations, or similar rubric-driven workAbility to interpret complex guidelines and apply consistent decisions at scaleComfort with web-based annotation tools, task queues, and documentation of edge casesStrong written English for rationale-based RLHF comparisons and clear communicationBenefitsCompetitive hourly rate: $30–$50/hrFully remote, full-time structured workflows with calibration and quality checkpointsWork on real-world LLM evaluation and multimodal annotation programs

📌 Remote Data Annotation Jobs Madrid (Sevilla)
🏢 Rex.zone
📍 Sevilla