Job Description
- Design and build distributed scraping architectures that can scale across thousands of domains
- Reverse engineer APIs and mobile apps using tools like Frida, mitmproxy, Charles Proxy, burp and many more.
- Detect common anti-bot measures (like JavaScript checks or browser behavior detection) to build crawlers that adjust to appear more human-like
- Lead technical innovation in scraping: identify edge cases, prototype new approaches, and share knowledge with the team
- Set up anomaly detection and monitoring to detect blockers or failures early

Qualifications
What You Bring
- 5+ years of experience with Python and large-scale scraping projects
- Deep understanding of anti-bot technologies and how to bypass them, fingerprinting, WAFs, JavaScript challenges, etc
- Experience working with Scrapy, Requests, httpx, Selenium, Playwright, or custom headless browser drivers
- Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection, and TLS tweaking
- Strong debugging skills across browser, HTTP traffic,



and device-level interactions
- You think independently and enjoy solving problems no one has solved before
- Familiarity with structured storage formats (Parquet, Avro), or real-time stream processing
- Good knowledge of databases like MongoDB, InfluxDB, and Redis, including how they work and how to optimize them

Bonus Skills
- Experience with real-device simulation for mobile scraping
- Comfortable building and maintaining distributed task queues (Celery, Kafka, etc.)
- Experience scraping high-security platforms or mobile-first environments
- Exposure to Linux or cloud environments (AWS, GCP, etc.)
- Basic understanding of Kubernetes, Cloud Run, and Cloud Functions to deploy and orchestrate scraping workloads

Additional Information
Why Join NIQ Digital shelf
- You'll have ownership and freedom to experiment, backed by a team that thrives on innovation
- You'll work with people who are truly passionate abou

📌 Senior Software Engineer (Madrid)
🏢 NielsenIQ
📍 Madrid