Commit Graph

430 Commits

Author SHA1 Message Date
Karim shoair fd9fc83c6c build: pump version up and deps 2025-12-17 00:04:24 +02:00
Karim shoair aed34e43b5 fix(fetchers): better handling for infinite network_idle 2025-12-03 03:11:30 +02:00
Karim shoair 16ff07a287 build: Pump version up and mcp 2025-12-03 02:20:18 +02:00
Karim shoair ac2d4d3833 fix: use the correct signature for the stealthy_fetch shortcut 2025-12-02 02:15:42 +02:00
Karim shoair 0e1f3b9b05 fix(fetchers): Add backward compatibility for custom_config argument 2025-11-26 02:23:17 +02:00
Karim shoair 9b3309f825 fix(shell): use correct argument name in signatures 2025-11-26 02:22:29 +02:00
Karim shoair 880b144af0 refactor(browser fetchers): Make all the type hints dynamic + Faster validation
+ Also renamed `custom_config` to `selector_config` so it matches the session class.
2025-11-25 22:03:45 +02:00
Karim shoair 8940bdeb55 fix(shell): dynamically build the signature of shortcuts after last changes 2025-11-25 21:23:06 +02:00
Karim shoair 04a612e64e docs: fix docstrings for all requests methods 2025-11-25 21:05:21 +02:00
Karim shoair 5376c97996 fix(fetcher): Remove non-keywords arguments 2025-11-25 20:25:09 +02:00
Karim shoair f0b01fc253 refactor(browser fetchers): Make all the type hints dynamic + Faster validation
Made the code shorter by an additional ~200 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-25 20:18:36 +02:00
Karim shoair 9e85311ff6 fix(validators): Fix overriding session values with request ones 2025-11-25 19:48:09 +02:00
Karim shoair 74f24d60f6 fix(fetcher): Replace vars logic with one that doesn't require dict attribute 2025-11-24 13:04:12 +02:00
Karim shoair c487faf4fd pref(requests): Optimizing memory by specifiying slots 2025-11-23 21:50:54 +02:00
Karim shoair 357b170e1c refactor(requests): Make all the type hints dynamic
Made the code shorter by about ~500 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-23 20:48:51 +02:00
Karim shoair 2bae3261a0 build: Pumping version and deps up
This also updates the Maxmind database for camoufox
2025-11-22 21:41:47 +02:00
Karim shoair 170205599d fix(parser): Better approach for web pages where the encoding is not always correctly declared
Fixes #110 and avoids defaulting to a specific encoding like #111
2025-11-22 20:24:55 +02:00
Karim shoair fb80beba04 feat(TextHandler): Add argument to clean method to remove html entities 2025-11-17 01:36:06 +02:00
Karim shoair 3565f9d500 feat(fetcher): Make impersonate able to randomize fingerprint 2025-11-16 16:33:58 +02:00
Karim shoair c8b9a04e87 docs: making some adjustments 2025-11-14 02:22:59 +02:00
Karim shoair d018d6e438 fix: fixing the response logic 2025-11-10 03:03:05 +02:00
Karim shoair ff19dd5bf5 feat: replace raw_response property with smart content detection 2025-11-10 02:39:45 +02:00
Karim shoair df073c4680 fix: added raw_response to separate collected response from rendered response 2025-11-08 20:09:40 +02:00
Karim shoair e20882af5d style: removing dead code from the parser 2025-11-08 20:08:23 +02:00
Karim shoair 42a8b43207 build: Pump version up 2025-11-08 20:07:26 +02:00
Karim shoair fc4a809985 fix(cloudflare): Limit the waiting period for websites that captcha doesn't disappear after solving
Also solves #100
2025-10-27 05:13:12 +03:00
Karim shoair d4a2507666 refactor(validation): All browser based fetchers are ~8-15% faster now in most cases 2025-10-27 04:48:45 +03:00
Karim shoair 13b9a4668c feat(DynamicSession): New option to add extra browser flags 2025-10-26 16:53:27 +03:00
Karim shoair 59ec6be201 refactor(fetchers): Less duplicated code and better handling for unstable websites
- Websites that never finish loading their requests won't crash the code now if you used `network_idle` with them
- Fixed a typo in `load_dom` in DynamicSession's async_fetch
- Removed dead code
2025-10-26 05:04:19 +03:00
Karim shoair 74fa45daea fix: Addressing the issue of collecting response after page_action in #100 2025-10-26 00:41:45 +03:00
Karim shoair a54ceb3335 build: Pumping up the deps and the version 2025-10-19 15:38:46 +03:00
Karim shoair 9c7114219b fix(solver): Improve the logic a bit 2025-10-12 06:41:04 +03:00
Karim shoair a38b92cc87 feat(Dynamic fetcher): Add additional_args option to customize browser as in StealthyFetcher 2025-10-12 05:07:22 +03:00
Karim shoair c34e10faed feat(fetchers): Add argument to control the user data directory 2025-10-12 04:48:59 +03:00
Karim shoair 8c995c936d fix(CF solver): further improve the detection 2025-10-12 04:15:16 +03:00
Karim shoair 073c4910d4 fix(fetchers): More accurate collection of the final response
This will make the `response.body` return the raw response for text/json responses as well
2025-10-12 04:12:18 +03:00
Karim shoair 6bfa266c90 fix(camoufox): Fix a typo in the async solver 2025-10-12 03:07:18 +03:00
Karim shoair 0dffa3bbba docs: removing unnecessary docstrings
They are internal functions, not user-facing ones
2025-10-06 14:24:16 +03:00
Karim shoair 9cae97b177 fix(requests): Fix multiple configuration issues 2025-10-06 06:53:09 +03:00
Karim shoair e53ce3f749 refactor: Making the requests logic better and cleaner
Also, solve some bugs along the way and a cleaner fix for #97
2025-10-06 05:00:41 +03:00
Karim shoair f4ce433ff3 build: pump up the version and deps 2025-10-05 04:08:19 +03:00
Karim shoair debe03256b refactor: Making all the codebase acceptable by PyRight
Also fixes #97
2025-10-05 04:03:39 +03:00
Karim shoair e149c715dd feat(CF solver): better implementation to help with all custom implementations of turnstile 2025-10-02 20:05:39 +03:00
Karim shoair 5127d386c9 style(Fetcher): correcting type hints and annotation 2025-10-01 06:18:23 +03:00
Karim shoair 849913ad77 style: Correcting return type annotation for Fetcher/AsyncFetcher (#93) 2025-10-01 04:17:59 +03:00
Karim shoair eedfa855ab fix: Fixes for the type checking in the main init file
and a bit of cleaning
2025-10-01 03:50:39 +03:00
Karim shoair 4135dd86b8 refactor: Restructure the fetchers code to not use more memory than needed
Also fixes: #92
2025-10-01 03:48:43 +03:00
Karim shoair f6c122b87f style: Removing dead code/docstrings 2025-09-29 05:09:23 +03:00
Karim shoair 8ad7cd6343 docs: Update all pages/docstrings to reflect recent changes 2025-09-29 03:57:40 +03:00
Karim shoair 840aef6884 build: Pump up version and deps 2025-09-28 22:04:09 +03:00