Who you are
- 8+ years of software engineering experience, including significant experience operating at senior or Staff-level scope
- Deep hands-on experience building and evaluating systems based on LLMs and modern AI tooling
- Strong software engineering fundamentals and the ability to rapidly build high-quality experimental systems
- Experience building agentic or multi-step AI systems involving tool use, orchestration, state, retrieval, or external integrations
- Strong knowledge of cloud infrastructure, preferably AWS, and the ability to run experimental workloads securely and cost-consciously
- Experience with observability, telemetry, testing, and benchmarking of complex systems
- Ability to reason about system architecture, reliability, scalability, asynchronous workflows, and distributed components where relevant
- Track record of designing experiments or benchmarks that influenced meaningful technical decisions
- Track record designing and running benchmarks that compare AI models and tools under real constraints
- Experience constructing evaluation datasets:



task selection, labelling, holdout discipline, and keeping a set useful as models improve
- Working knowledge of LLM-as-judge methods and their failure modes, alongside human evaluation, inter-annotator agreement, and a view on when each is appropriate
- Able to reason about statistical significance on small samples, and to state confidence honestly rather than over-reading a result
- Familiarity with regression tracking, telemetry and versioning, so that a result stays reproducible months later
- Able to turn ambiguous "we should try this new thing" ideas into well-scoped evaluation plans with clear hypotheses and metrics
- Comfortable making trade-off calls across quality, latency, cost and vendor lock-in, and documenting them clearly
- Experience writing short, opinionated decision memos that help others move fast
- Can explain technical results to non-specialists in concrete, concise terms
- Expe

📌 Staff AI Engineer (Barcelona)
🏢 dLocal
📍 Barcelona