ОписаниеNIQ is a consumer intelligence company that analyzes consumer buying behavior and identifies growth opportunities. Its platforms combine retail data and advanced analytics to deliver consumer insights. NIQ operates in more than 100 markets and covers over 90% of the world’s population.Задачи Design and build distributed scraping architectures that scale across thousands of domainsReverse engineer APIs and mobile apps using tools such as Frida, mitmproxy, Charles Proxy, and BurpDetect anti-bot measures, including Java Script checks and browser behavior detection, and build crawlers that adapt to appear more human-likeLead technical innovation in scraping, identify edge cases, prototype new approaches, and share knowledge with the teamSet up anomaly detection and monitoring to identify blockers or failures early Требования 5+ Years of experience with Python and large-scale scraping projectsDeep understanding of anti-bot technologies and how to bypass them, including fingerprinting, WAFs, and Java Script challengesExperience with Scrapy, Requests, httpx, Selenium, Playwright,



or custom headless browser driversSkilled in handling residential, datacenter, and rotating proxies, session management, cookie injection, and TLS tweakingStrong debugging skills across browser, HTTP traffic, and device-level interactionsIndependent problem-solving skills and enjoyment of tackling unsolved problemsGood knowledge of Mongo DB, Influx DB, and Redis, including their operation and optimizationБудет плюсом: familiarity with Parquet or Avro, real-time stream processing, real-device simulation for mobile scraping, distributed task queues such as Celery or Kafka, scraping high-security platforms or mobile-first environments, Linux or cloud environments such as AWS or GCP, basic Kubernetes, Cloud Run, and Cloud Functions knowledge for deploying and orchestrating scraping workloads Условия Versátil working hoursFlexible working environmentVolunteer time offLinked In LearningEmployee Assistance Program (EAP) #J-18808-Ljbffr

📌 Software Engineer (Web Scraping / Data Acquisition) (Madrid)
🏢 NielsenIQ
📍 Madrid