Project Title: Private Advanced Web Scraper with Live Updates Description: I need a custom-built web scraper for my personal use. This isn’t for resale or distribution — it’s a private tool that I’ll be the only one using. What I’m looking for: Config-based scraper: I should be able to adjust a JSON config to target different sites (CSS selectors for fields, pagination, next button, etc.). Live status logging: Console updates + simple log file so I can watch progress while it runs. Checkpointing: If the scraper stops, I should be able to resume without starting over. Deduplication: No duplicate items. Output formats: Save results continuously to JSONL and easily convert/export to CSV. Polite scraping: Built-in delays, retries, user-agent handling. Optional: A simple HTML status page that auto-refreshes so I can track progress visually. Deliverables: Main scraper script (Python preferred, Requests + BeautifulSoup or Selenium if needed) Example config file so I can swap in my own targets Helper to convert JSONL → CSV README for setup and usage Timeline: By Monday