In this role you will help optimize data collection as a Python Developer in our scraping team. You will design, implement, and document robust Scrapy spiders to withstand website changes while preserving existing functionality. You’ll ensure high data quality through code reviews and validation, and build sophisticated crawlers that navigate anti-bot measures. You will contribute to major developments across codebases and share knowledge through documentation and training. This is a chance to shape scalable data pipelines for real-time market visibility in a fast-growing, startup-minded environment.
Compensaciones / Beneficios
- Versátil working environment
- Volunteer time off
- LinkedIn Learning
- Employee-Assistance-Program (EAP)
Responsabilidades
- Design, implement and document Scrapy spiders for robust data collection
- Perform code reviews and data validation to ensure quality
- Develop crawlers that handle anti-bot measures using HTTP and browser techniques
- Architect and contribute to new developments across multiple codebases with documentation and training
- Collaborate with cross-functional teams to share knowledge and enable tooling improvements
Requisitos principales
- Master in Computer Science, IT, or related field
- At least 3 years of professional software engineering experience
- Knowledge of BeautifulSoup or Scrapy
- Familiarity with HTML and JavaScript, understanding of SPAs
- Experience with RESTful and/or GraphQL APIs
- Hands-on experience with Django, FastAPI or similar Python frameworks
- Scrapy knowledge is a plus
- Experience with time series databases like InfluxDB is advantageous
- Strong skills in Docker, Git, pandas, regex, Linux, and bash scripting
- Proven experience with AWS, GCP, or Azure