Crowley Shopee Data Scraper & Real-Time Product Monitoring System

Замовник: AI | Опубліковано: 08.10.2025
Бюджет: 250 $

Project Description: I am looking for a skilled freelancer to develop a Crowley automation system that continuously scrapes product data from the Shopee website. The system must ensure that all product information is always up to date in real time, maintaining a 99% accuracy rate. The Crowley should operate in an infinite loop, starting from page 0, and automatically capturing product links, references, and related data. It must collect all products available on Shopee, including details such as product titles, prices, images, descriptions, sellers, categories, and especially data on top-selling products. Additionally, the freelancer must build an API that retrieves and serves the collected data efficiently. The API should allow filtered queries (for example, by product category, keyword, or price range). All of this should run autonomously on a server, ensuring continuous operation and data consistency without manual intervention. Requirements: Advanced experience in web scraping and data automation (preferably with Python). Knowledge of handling anti-bot protection, proxies, and rotating IPs. Experience creating and maintaining APIs. Understanding of database design and optimization for storing large volumes of data. The system must be efficient, stable, and scalable. Goal: To have a fully automated Crowley that keeps the Shopee product database constantly updated and accessible through an API with near-perfect accuracy.