Please submit your CV in English and indicate your level of English proficiency.

Puede obtener más detalles sobre la naturaleza de esta vacante y lo que se espera de los solicitantes leyendo la información a continuación.

Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems. Participation is project-based, not permanent employment.

About the Role

You’ll design coding tasks that challenge frontier AI coding agents. Each task is a self-contained Docker environment with a broken piece of software; an AI agent attempts the fix; automated tests verify the outcome. Your deliverable is the full task package: broken code, tests, instructions, and a reference solution proving the task is solvable.

Responsibilities:
- Invent a realistic developer scenario — a real bug, a broken ETL, a missing feature — not a toy problem.
- Build a reproducible Docker environment with pinned dependencies.
- Write a pytest that verifies outcomes, not specific commands — deterministic, non-flaky,



and does not leak the fix.
- Write an that reads like a Jira ticket a developer would receive.
- Write a reference proving the task is solvable.
- Calibrate difficulty so current state-of-the-art agents solve the task 20–60% of the time.
- Iterate based on feedback from expert QA reviewers.
- Later: review other authors’ tasks as a QA reviewer.

Not in scope
- Data labeling, prompt engineering.
- Production code to ship — you design problems and verification for AI agents.
- Leetcode puzzles — scenarios must look like real developer work.
- Not every candidate task ships — quality over quantity.

Requirements
- 3+ years of production software development in one backend stack — Python, Go, Node.js, Java, or Rust. Depth in one stack beats breadth.
- Python + pytest fluency — required regardless of primary stack. The task harness is pytest-based even when the broken app is in another language. Fixtures, p

📌 Software Engineering Evaluation Specialist (España)
🏢 Mindrift
📍 España