Real-Time Gym Equipment Web Scraper

Замовник: AI | Опубліковано: 23.11.2025
Бюджет: 5000 $

Project Summary We are a Fitness Equipment resale company looking to gain a significant sourcing advantage by implementing a highly reliable, custom web scraping bot. The goal is to monitor Facebook Marketplace for newly posted listings of commercial gym equipment in real-time. The core requirement is a production-ready, highly robust, and anti-detection focused solution. This is not a hobby project; we require an expert capable of building a stable, long-term system. Required Expertise (Technical Must-Haves) • Language: Expert proficiency in Python 3. • Web Automation: Demonstrated experience with modern browser automation frameworks (Playwright or advanced Selenium) to handle dynamic, JavaScript-heavy content. Simple requests or BeautifulSoup solutions are not acceptable. • Anti-Detection Strategy: Deep understanding and proven ability to implement IP rotation/proxies, User-Agent rotation, and session management (cookie persistence, handling login flows) to avoid rate limiting and account bans. • Data Storage: Experience integrating with a scalable database (e.g., PostgreSQL, MongoDB) or a simple cloud storage solution (Google Sheets, Airtable) for clean data output. Key Bot Functionality & Deliverables The final bot system must achieve the following: 1. Real-Time Data Acquisition • Target: Facebook Marketplace search results filtered by specific keywords (e.g., "Used Nautilus Machine"), category, price range, and a defined geographical radius We purchase everywhere in the Unites States. • Scraping Frequency: Must be capable of running checks every 1-5 minutes without failure to ensure true real-time sourcing. • Data Points to Extract: • Listing Title, Full URL • Price (parsed and normalized into a clean numeric format) • Location (City/ZIP code) • Date/Time Posted (Crucial for filtering new listings) • Seller Profile URL • Primary Image URL 2. Filtering & Alert System (Core Value) • De-duplication: Automatically ignore listings that have already been processed and saved. • Conditional Alert: The bot must identify listings posted within the last [e.g., 5 minutes] • Notification: Send instant alerts via SMS/Text, containing the Price, Title, and Link of the hot listing. 3. Deployment and Maintenance • Hosting: We require assistance deploying the bot to a cloud environment (e.g., AWS, DigitalOcean) so it runs 24/7. • Source Code: Full ownership and delivery of clean, commented, and well-structured source code (Python). • Error Monitoring: The script must include robust logging and an alert system to notify us immediately if the scraper fails or receives an IP block, minimizing downtime. Submission Requirements To demonstrate your expertise, please address the following in your proposal: 1. Describe your specific strategy for avoiding detection and ensuring stability on a highly monitored site like Facebook. 2. Provide brief details on the most complex web scraping project you have successfully deployed and maintained. 3. Confirm your ability to assist with cloud hosting and ongoing maintenance (paid hourly or via a monthly retainer). Thank You for your consideration!