I need a scraper that reliably pulls live trading data from a web based trading platform. The scraper should run as frequently as the target site and practical rate-limits allow "ideally near real time" and quietly recover from captchas, session time-outs, and any layout changes without manual fixes. Because this data will feed other internal dashboards and models, I’m open to your recommendation on the optimal delivery mechanism. Whether that’s streaming the results into a lightweight database, pushing updates through an internal REST endpoint, or simply appending to time stamped CSVs, the key is low latency, easy downstream integration, and strong fault tolerance. Solution must be well commented, maintainable code (Python with requests, Selenium, or Scrapy is fine, but I’m flexible) and include clear setup instructions Once 100% complete and all bugs have been fixed, I’ll test it against live data; if it can maintain uninterrupted collection and accurate data over a full 24 hrs trading period, I’ll sign off.