- Implement RequestCache class with TTL-based expiration (5 min default) - Cache results when include_comments=false for faster repeated requests - Skip caching when comments requested as they change frequently - Pre-inject helper functions once per page load via _inject_helpers() - Batch DOM operations: expandAllComments() before extraction - Single JavaScript call getComments(maxDepth, maxCount) for nested structure - Reduces JS overhead by 50%+ and eliminates repeated script parsing |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| cache.py | ||
| selenium_scrapers.py | ||