Overview In this role you will design and operate scalable distributed scraping systems to extract high-quality data from thousands of domains. You will work within the Scraping and R&D; team to push innovative approaches and share knowledge, collaborating with Customer Success, Operations, and Data. You'll tackle anti-bot challenges and build monitoring to prevent blockers, contributing to fast, scalable visibility for retailers and brands.

This is a hands-on, problem‐solving role for someone who loves turning difficult data problems into reliable solutions.

Compensaciones / Beneficios Versátil working environment

Volunteer time off

LinkedIn Learning

Employee-Assistance-Program (EAP)

Responsabilidades Design and build scalable distributed scraping architectures for thousands of domains

Reverse engineer APIs and mobile apps using tools like Frida, mitmproxy, Charles Proxy, burp

Detect and bypass anti-bot measures to create human-like crawlers

Lead technical innovation in scraping, prototype approaches, share knowledge with the team

Set up anomaly detection and monitoring to catch blockers or failures early





Requisitos principales 5+ years of Python experience and involvement in large-scale scraping projects

Deep understanding of anti-bot technologies and bypass techniques, fingerprinting, WAFs, JavaScript challenges

Experience with Scrapy, Requests, Selenium, Playwright or similar headless drivers

Proficiency with proxies (residential, datacenter, rotating), session management, cookie injection, TLS tweaking

Strong debugging skills across browser, traffic, and device-level interactions

Independent problem solver with a track record of solving new problems

Familiarity with Parquet, Avro or real-time stream processing

Knowledge of databases like MongoDB, InfluxDB, and Redis and how to optimize them

Independent thinking

Strong problem-solving abilities

Self-motivation and proactivity

Python for large-scale scraping

Anti-bot bypass techniques, fingerprinting, WAFs, JavaScript challenges

Scrapy, Requests, Selenium, Playwright or headless drivers

📌 Senior Software Engineer (Web Scraping, Data acquisition, Data collection) (Barcelona)
🏢 NielsenIQ
📍 Barcelona