Job Description

Design and build distributed scraping architectures that can scale across thousands of domains
Reverse engineer APIs and mobile apps using tools like

Frida ,

mitmproxy ,

Charles Proxy, burp and many more.
Detect common anti-bot measures (like JavaScript checks or browser behavior detection) to build crawlers that adjust to appear more human-like
Lead technical innovation in scraping: identify edge cases, prototype new approaches, and share knowledge with the team
Set up anomaly detection and monitoring to detect blockers or failures early
Qualifications

What You Bring

5+ years of experience with

Python

and large-scale scraping projects
Deep understanding of

anti-bot technologies

and how to bypass them, fingerprinting, WAFs, JavaScript challenges, etc
Experience working with

Scrapy ,

Requests ,

httpx ,

Selenium ,

Playwright , or custom headless browser drivers
Skilled at handling proxies (residential, datacenter, rotating), session management, cookie injection, and TLS tweaking
Strong debugging skills across browser, HTTP traffic,



and device-level interactions
You think independently and enjoy solving problems no one has solved before
Familiarity with structured storage formats (Parquet, Avro), or real-time stream processing
Good knowledge of databases like MongoDB, InfluxDB, and Redis, including how they work and how to optimize them
Bonus Skills

Experience with real-device simulation for mobile scraping
Comfortable building and maintaining distributed task queues (Celery, Kafka, etc.)
Experience scraping high-security platforms or mobile-first environments
Exposure to Linux or cloud environments (AWS, GCP, etc.)
Basic understanding of Kubernetes, Cloud Run, and Cloud Functions to deploy and orchestrate scraping workloads
Additional Information

Why Join NIQ Digital shelf

You'll have

ownership

and

freedom

to experiment, backed by a team that thrives on innovation
You'll work with people who are truly passionate about data and kno

📌 Senior Software Engineer (Madrid)
🏢 NielsenIQ
📍 Madrid