freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,321 Commits 1 Branch 0 Tags
0bbe62fc7f7011e28dc3d535d36d32944bd30d9a
Commit Graph
3 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Abdullah 0bbe62fc7f feat(spiders): implement RobotsTxtManager with concurrent fetch deduplication 2026-04-03 15:08:33 +02:00
Abdullah 1d15349e07 feat(deps): add protego for robots.txt parsing and fix pyright type error in static.py 2026-04-03 15:08:33 +02:00
Abdullah 7ba333c3c9 perf(session): improve page management and speed up execution 
Close pages immediately after tasks finish to reduce resource usage and
improve overall performance. Update corresponding test files to reflect
the new page lifecycle management.
 2025-09-17 18:49:45 +03:00