Job Title: AI Policy Reviewer

Location: Remote (Worldwide)

Job Summary: The AI Policy Reviewer is responsible for evaluating AI-generated and user-generated content to ensure compliance with internal governance standards, regulatory requirements, and responsible AI principles. This role plays a key part in safeguarding model integrity by reviewing outputs for safety risks, bias, misinformation, harmful content, and policy violations, while ensuring consistent enforcement of AI usage guidelines.

Responsibilities

Review and score AI-generated responses against detailed policy rubrics. Assess outputs for safety, truthfulness, fairness, and alignment with community guidelines.
Act as a quality assurance checkpoint for automated systems. Identify instances where the AI misinterprets policy (e.g., being over-sensitive and censoring benign content, or under-sensitive and allowing harmful content).
Handle complex “edge cases” where policy application is ambiguous. Make nuanced judgement calls regarding context, satire, or emerging risks that the AI model struggles to process.




Analyze and review data to identify systematic flaws in the AI’s reasoning. Report patterns of bias, hallucination, or policy gaps to the Product and Engineering teams.
Collaborate with Policy teams to test and refine evaluation rubrics. Provide feedback on whether current policies are “teachable” to AI models or if they require human-only judgement.
Participate in adversarial testing (red teaming) by attempting to “jailbreak” the model or provoke unsafe responses to identify vulnerabilities before launch.
Work closely with Machine Learning Engineers to explain the “why” behind your ratings, helping them adjust model behavior.
Write high-quality examples (prompts and idóneo responses) that’s serve as “golden sets” for training the AI on how to handle difficult policy scenarios.

Requirements

Minimum of 3 years of professional experience in Trust & Safety Operations, Content Policy, Risk Analysis, o

📌 A.I Policy Reviewer (España)
🏢 Odixcity Consulting
📍 España