Experteer Overview
In this role you will design and scale distributed scraping architectures to extract high-quality data at speed and scale for NIQ’s global customers. You join a cross-functional Scraping and Ru0026amp;D team that values experimentation and bold ideas, collaborating with Customer Success, Operations, and Data to transform raw data into actionable insights. You will tackle anti-bot challenges, prototype innovative approaches, and build monitoring to catch blockers early. This is a chance to shape our tooling and approach to large-scale data scraping in a fast-moving environment.

Compensaciones / Ventajas
• Design and build distributed scraping architectures scalable across thousands of domains
• Reverse engineer APIs and mobile apps using Frida, mitmproxy, Charles Proxy, Burp, etc.
• Detect and adapt to anti-bot measures to keep crawlers human-like
• Lead technical innovation in scraping by identifying edge cases and prototyping new approaches




• Set up anomaly detection and monitoring to identify blockers or failures early

Responsabilidades
• 5+ years of Python experience with large-scale scraping projects
• Deep understanding of anti-bot technologies and bypass techniques
• Experience with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless drivers
• Proficiency in handling proxies (residential, datacenter, rotating), session management, cookie injection, TLS tweaking
• Strong debugging skills across browser, HTTP traffic, and device-level interactions
• Independent thinker with strong problem-solving abilities
• Familiarity with structured storage formats (Parquet, Avro) or real-time stream processing
• Good knowledge of databases like MongoDB, InfluxDB, and Redis and how to optimize them

Requisitos principales
• Flexible working environment
• Volunteer time off
• LinkedIn Learning
• Employee-Assistance-Program (EAP)

📌 Senior Web Scraper (Scrapy) (Madrid)
🏢 NielsenIQ
📍 Madrid