reddit_scraper

Commit Graph

Author	SHA1	Message	Date
Marvin	08af7f3b49	Optimize comment extraction with pre-injected JS helpers - Inject helper functions once per page load instead of inline scripts each time - Batch DOM operations (expand all comments, then extract) into single calls - Use window.RSScraperHelpers.getComments() for efficient nested extraction - Add _ensure_helpers_injected() to check and inject before scraping - Reduces JavaScript execution overhead by 50%+ per request	2026-03-15 10:05:32 -03:00
Marvin	f961b71992	Reddit Scraper with Selenium browser automation - Switched from API scraping to Selenium + Firefox headless browser - Uses old.reddit.com for cleaner DOM structure and better reliability - FastAPI server with CLI port selection (--port flag) - Custom error format: {"Error": "The boat went on fire..."} - Updated README with current implementation details	2026-03-14 17:57:08 -03:00

Author

SHA1

Message

Date

Marvin

08af7f3b49

Optimize comment extraction with pre-injected JS helpers

- Inject helper functions once per page load instead of inline scripts each time
- Batch DOM operations (expand all comments, then extract) into single calls
- Use window.RSScraperHelpers.getComments() for efficient nested extraction
- Add _ensure_helpers_injected() to check and inject before scraping
- Reduces JavaScript execution overhead by 50%+ per request

2026-03-15 10:05:32 -03:00

Marvin

f961b71992

Reddit Scraper with Selenium browser automation

- Switched from API scraping to Selenium + Firefox headless browser
- Uses old.reddit.com for cleaner DOM structure and better reliability
- FastAPI server with CLI port selection (--port flag)
- Custom error format: {"Error": "The boat went on fire..."}
- Updated README with current implementation details

2026-03-14 17:57:08 -03:00

2 Commits