a17bb1d976
feat(browsers): Add option to retry tabs that gives errors
2026-01-11 23:40:26 +02:00
d5b9ed11b2
fix(browsers): Close pages that give error
...
This was causing the browser and the whole code to stand still.
2026-01-11 18:34:13 +02:00
c5bc82deb8
fix(browsers): solve a bug with setting referer on the request level
2026-01-11 03:21:03 +02:00
08f3e6efad
fix: add missing line
2026-01-08 01:01:00 +02:00
6a73f9dcd7
refactor: internal API changes to be easily used as indicators for spiders
2026-01-08 00:52:06 +02:00
4947152be2
docs: change docstring style for some methods
2026-01-01 21:06:35 +02:00
f6f0ed4b37
fix(async stealth): wait the coroutine of the bounding box
2025-12-31 00:19:53 +02:00
ee13af9516
build: update deps
...
and docstrings
2025-12-29 16:08:51 +02:00
5e3845f060
docs: shorten docstrings for some functions
2025-12-28 21:56:28 +02:00
9bfe744840
fix: remove forgotten unused code
2025-12-28 00:14:47 +02:00
1b838eabd0
fix: update the cookies type hint for the mcp server
2025-12-28 00:08:44 +02:00
09bbd416b8
fix: remove old proxy logic
...
When I was caching it with the rest of the data
2025-12-28 00:02:50 +02:00
842718ba94
fix: update shell signatures for the autocompletion
...
Added missing fields and reordered classes as well for easier comparisons
2025-12-27 14:30:48 +02:00
cfc667f7dc
refactor(fetchers)!: Replace Camoufox with patchright and many optimizations
...
- DynamicFetcher became 20% faster
- StealthyFetcher became 99% faster
- Scrapling size decreased
- Code became ~400 lines shorter
- Most importantly, scrapling is more stable and reliable now.
- Less confusing for new users.
- More...
2025-12-26 03:21:13 +02:00
f9f34e6ee9
fix: improve the handling of useragent in chrome fetcher
2025-12-24 23:08:30 +02:00
ecb97d3fd6
docs: add docs for timezone_id argument
2025-12-18 01:16:22 +02:00
39f544eae4
feat(dynamicSession): add the option to set timezone of the browser
2025-12-18 01:13:31 +02:00
b507e4d4a0
refactor(fetchers): rename internal api
...
To make it easier to use to use sessions outside `with` context
2025-12-17 01:27:40 +02:00
aed34e43b5
fix(fetchers): better handling for infinite network_idle
2025-12-03 03:11:30 +02:00
f0b01fc253
refactor(browser fetchers): Make all the type hints dynamic + Faster validation
...
Made the code shorter by an additional ~200 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-25 20:18:36 +02:00
9e85311ff6
fix(validators): Fix overriding session values with request ones
2025-11-25 19:48:09 +02:00
c487faf4fd
pref(requests): Optimizing memory by specifiying slots
2025-11-23 21:50:54 +02:00
357b170e1c
refactor(requests): Make all the type hints dynamic
...
Made the code shorter by about ~500 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-23 20:48:51 +02:00
df073c4680
fix: added raw_response to separate collected response from rendered response
2025-11-08 20:09:40 +02:00
fc4a809985
fix(cloudflare): Limit the waiting period for websites that captcha doesn't disappear after solving
...
Also solves #100
2025-10-27 05:13:12 +03:00
d4a2507666
refactor(validation): All browser based fetchers are ~8-15% faster now in most cases
2025-10-27 04:48:45 +03:00
13b9a4668c
feat(DynamicSession): New option to add extra browser flags
2025-10-26 16:53:27 +03:00
59ec6be201
refactor(fetchers): Less duplicated code and better handling for unstable websites
...
- Websites that never finish loading their requests won't crash the code now if you used `network_idle` with them
- Fixed a typo in `load_dom` in DynamicSession's async_fetch
- Removed dead code
2025-10-26 05:04:19 +03:00
74fa45daea
fix: Addressing the issue of collecting response after page_action in #100
2025-10-26 00:41:45 +03:00
9c7114219b
fix(solver): Improve the logic a bit
2025-10-12 06:41:04 +03:00
a38b92cc87
feat(Dynamic fetcher): Add additional_args option to customize browser as in StealthyFetcher
2025-10-12 05:07:22 +03:00
c34e10faed
feat(fetchers): Add argument to control the user data directory
2025-10-12 04:48:59 +03:00
8c995c936d
fix(CF solver): further improve the detection
2025-10-12 04:15:16 +03:00
073c4910d4
fix(fetchers): More accurate collection of the final response
...
This will make the `response.body` return the raw response for text/json responses as well
2025-10-12 04:12:18 +03:00
6bfa266c90
fix(camoufox): Fix a typo in the async solver
2025-10-12 03:07:18 +03:00
e53ce3f749
refactor: Making the requests logic better and cleaner
...
Also, solve some bugs along the way and a cleaner fix for #97
2025-10-06 05:00:41 +03:00
debe03256b
refactor: Making all the codebase acceptable by PyRight
...
Also fixes #97
2025-10-05 04:03:39 +03:00
e149c715dd
feat(CF solver): better implementation to help with all custom implementations of turnstile
2025-10-02 20:05:39 +03:00
eedfa855ab
fix: Fixes for the type checking in the main init file
...
and a bit of cleaning
2025-10-01 03:50:39 +03:00
f6c122b87f
style: Removing dead code/docstrings
2025-09-29 05:09:23 +03:00
8ad7cd6343
docs: Update all pages/docstrings to reflect recent changes
2025-09-29 03:57:40 +03:00
716a5c3572
feat(cf solver): Make solver able to handle Turnstile and interstitial
2025-09-28 02:53:51 +03:00
82b09620b2
perf: Improving all browser-based fetches by ~15%
2025-09-20 06:20:23 +03:00
970ad40404
perf: Speed up sync fetches by ignoring number of pages checking
2025-09-20 04:31:57 +03:00
67db865aee
feat(DynamicFetcher): Replace rebrowser with patchright
2025-09-20 00:23:29 +03:00
a9d05cc0ef
style: Removing dead code/docstrings and correcting type hints
2025-09-19 04:49:30 +03:00
7ba333c3c9
perf(session): improve page management and speed up execution
...
Close pages immediately after tasks finish to reduce resource usage and
improve overall performance. Update corresponding test files to reflect
the new page lifecycle management.
2025-09-17 18:49:45 +03:00
5683965367
fix(sessions): Fixing a bug with max pages counter
2025-09-16 01:47:31 +03:00
5bcd4c2927
fix: A workaround for Playwright issue with page content retrieving on windows
2025-09-15 21:34:20 +03:00
7a7097b775
fix: Stop removing browser default page
...
Looks like it's problematic with some systems so it doesn't worth the trouble
2025-09-15 13:43:47 +03:00
Karim shoair