Job Type: Adaptable Contract (Remote)
Job Summary: We are hiring senior security researchers, including faculty, research scientists, and experienced postdoctoral researchers, for a contract project that tests whether frontier AI models can carry out real research work in a command line environment. You will design research-grade tasks that a leading AI model cannot solve, then review how and why the model fails.
What you will do
• Design self-contained command line tasks drawn from the practice of machine learning research, such as implementing a method from a paper's description alone, locating the error in a flawed experimental setup, reproducing a reported result and explaining a discrepancy, implementing a custom loss function or optimizer correctly, and running a sound statistical comparison between two models.
• Build the task environment, including code, data, and dependencies, write a reference solution, and write automated tests that verify whether a solution is correct. Because results can vary between runs, tests need to be deterministic or use well-justified tolerances.
• Run your task against a frontier AI model, review its attempt with the same rigor you would apply to a paper under peer review, and refine the task until the failure reflects a real gap in the model's capability rather than ambiguity or trick wording.
• Work with reviewers to bring each task to acceptance.
Who we are looking for
• A PhD in security engineering, machine learning, computer science, statistics, or a closely related field.
• Industry or academic experience in machine learning research, in academia or industry. Years spent on doctoral research count toward this total.
• At least one publication, ideally peer-reviewed work at a recognized machine learning venue or journal.
• Deep working knowledge of Python and at least one major framework, such as PyTorch or JAX.
• Fluency in the Linux command line, shell scripting, Docker, and Git.
• Experience as a peer reviewer, or in
📌 Applied Security Researcher (España)
🏢 Cobalt
📍 España