- Implement RequestCache class with TTL-based expiration (5 min default)
- Cache results when include_comments=false for faster repeated requests
- Skip caching when comments requested as they change frequently
- Pre-inject helper functions once per page load via _inject_helpers()
- Batch DOM operations: expandAllComments() before extraction
- Single JavaScript call getComments(maxDepth, maxCount) for nested structure
- Reduces JS overhead by 50%+ and eliminates repeated script parsing
- Inject helper functions once per page load instead of inline scripts each time
- Batch DOM operations (expand all comments, then extract) into single calls
- Use window.RSScraperHelpers.getComments() for efficient nested extraction
- Add _ensure_helpers_injected() to check and inject before scraping
- Reduces JavaScript execution overhead by 50%+ per request
- Switched from API scraping to Selenium + Firefox headless browser
- Uses old.reddit.com for cleaner DOM structure and better reliability
- FastAPI server with CLI port selection (--port flag)
- Custom error format: {"Error": "The boat went on fire..."}
- Updated README with current implementation details
Marvin