Overview

In this role you will design and scale distributed scraping architectures to process billions of data points for retailers and brands. You will reverse-engineer APIs and mobile apps, tackle anti-bot defenses, and push technical innovation in scraping. You’ll implement monitoring to catch blockers early and collaborate with cross-functional teams to deliver high-quality data at speed. This is a hands-on, problem-solving role at the intersection of engineering and data insights, with a strong impact on visibility and decision-making for customers.

Compensaciones / Incentivos

- Flexible working environment
- Volunteer time off
- LinkedIn Learning
- Employee-Assistance-Program (EAP)

Responsabilidades

- Design and build scalable distributed scraping architectures for thousands of domains
- Reverse-engineer APIs and mobile apps using Frida, mitmproxy, Charles Proxy, burp, and similar tools
- Identify and circumvent anti-bot measures to create more human-like crawlers
- Lead technical innovation in scraping by prototyping approaches and sharing knowledge




- Set up anomaly detection and monitoring to detect blockers or failures early

Requisitos principales

- 5+ years of Python experience and large-scale scraping projects
- Deep understanding of anti-bot technologies and bypass techniques (fingerprinting, WAFs, JavaScript challenges)
- Experience with Scrapy, Requests, Selenium, Playwright, or custom headless drivers
- Proficiency in handling proxies (residential/datacenter/rotating), session management, cookie injection, TLS tweaking
- Strong debugging across browser, HTTP traffic, and device-level interactions
- Independent problem solver who enjoys tackling unusual challenges
- Familiarity with Parquet/Avro or real-time stream processing
- Knowledge of databases like MongoDB, InfluxDB, and Redis and optimization basics

- Independent thinker
- Problem-solving orientation
- Collaborative mindset
- Frida
- mitmproxy
- Charles Proxy

📌 Senior Software Engineer (Web Scraping, Data acquisition, Data collection) (Barcelona)
🏢 NielsenIQ
📍 Barcelona