Commit Graph

398 Commits

Author SHA1 Message Date
Karim shoair a38b92cc87 feat(Dynamic fetcher): Add additional_args option to customize browser as in StealthyFetcher 2025-10-12 05:07:22 +03:00
Karim shoair c34e10faed feat(fetchers): Add argument to control the user data directory 2025-10-12 04:48:59 +03:00
Karim shoair 8c995c936d fix(CF solver): further improve the detection 2025-10-12 04:15:16 +03:00
Karim shoair 073c4910d4 fix(fetchers): More accurate collection of the final response
This will make the `response.body` return the raw response for text/json responses as well
2025-10-12 04:12:18 +03:00
Karim shoair 6bfa266c90 fix(camoufox): Fix a typo in the async solver 2025-10-12 03:07:18 +03:00
Karim shoair 0dffa3bbba docs: removing unnecessary docstrings
They are internal functions, not user-facing ones
2025-10-06 14:24:16 +03:00
Karim shoair 9cae97b177 fix(requests): Fix multiple configuration issues 2025-10-06 06:53:09 +03:00
Karim shoair e53ce3f749 refactor: Making the requests logic better and cleaner
Also, solve some bugs along the way and a cleaner fix for #97
2025-10-06 05:00:41 +03:00
Karim shoair f4ce433ff3 build: pump up the version and deps 2025-10-05 04:08:19 +03:00
Karim shoair debe03256b refactor: Making all the codebase acceptable by PyRight
Also fixes #97
2025-10-05 04:03:39 +03:00
Karim shoair e149c715dd feat(CF solver): better implementation to help with all custom implementations of turnstile 2025-10-02 20:05:39 +03:00
Karim shoair 5127d386c9 style(Fetcher): correcting type hints and annotation 2025-10-01 06:18:23 +03:00
Karim shoair 849913ad77 style: Correcting return type annotation for Fetcher/AsyncFetcher (#93) 2025-10-01 04:17:59 +03:00
Karim shoair eedfa855ab fix: Fixes for the type checking in the main init file
and a bit of cleaning
2025-10-01 03:50:39 +03:00
Karim shoair 4135dd86b8 refactor: Restructure the fetchers code to not use more memory than needed
Also fixes: #92
2025-10-01 03:48:43 +03:00
Karim shoair f6c122b87f style: Removing dead code/docstrings 2025-09-29 05:09:23 +03:00
Karim shoair 8ad7cd6343 docs: Update all pages/docstrings to reflect recent changes 2025-09-29 03:57:40 +03:00
Karim shoair 840aef6884 build: Pump up version and deps 2025-09-28 22:04:09 +03:00
Karim shoair 2bad72b77a fix: correct http option type in the mcp server 2025-09-28 21:19:51 +03:00
Karim shoair 74f20d2a0e refactor: better implementation for the mcp mode 2025-09-28 20:14:04 +03:00
Karim shoair 4a661b4875 feat: Make mcp able to use http transport 2025-09-28 04:54:35 +03:00
Karim shoair 716a5c3572 feat(cf solver): Make solver able to handle Turnstile and interstitial 2025-09-28 02:53:51 +03:00
Karim shoair 3da806210b perf: General code restructure to not use more than needed memory 2025-09-23 18:34:54 +03:00
Karim shoair 63a46a8e0b fix(parser): An encoding issue with converting bytes to string on some encoding types
Removing that `invalid start byte` annoying bug
2025-09-23 18:32:56 +03:00
Karim shoair 82b09620b2 perf: Improving all browser-based fetches by ~15% 2025-09-20 06:20:23 +03:00
Karim shoair 970ad40404 perf: Speed up sync fetches by ignoring number of pages checking 2025-09-20 04:31:57 +03:00
Karim shoair 5164b801ec build: pump version up and update deps 2025-09-20 00:33:07 +03:00
Karim shoair 67db865aee feat(DynamicFetcher): Replace rebrowser with patchright 2025-09-20 00:23:29 +03:00
Karim shoair 2d704b2a8b fix(shell): Fixing a bug with content converting 2025-09-19 04:50:15 +03:00
Karim shoair a9d05cc0ef style: Removing dead code/docstrings and correcting type hints 2025-09-19 04:49:30 +03:00
Karim shoair bf5fe6d451 fix(shell): Fix view command edge case 2025-09-19 04:21:15 +03:00
Karim shoair 504ecdf7dc fix(shell): Fix parsing curl --data-raw 2025-09-19 04:20:16 +03:00
Karim shoair b197f8a5f9 fix(parser): Improve selectors re function 2025-09-18 13:56:37 +03:00
Abdullah 7ba333c3c9 perf(session): improve page management and speed up execution
Close pages immediately after tasks finish to reduce resource usage and
improve overall performance. Update corresponding test files to reflect
the new page lifecycle management.
2025-09-17 18:49:45 +03:00
Karim shoair 570303f938 fix(parser): Make html_content and prettify return strings not bytes (depends on the encoding) 2025-09-16 01:58:51 +03:00
Karim shoair 5683965367 fix(sessions): Fixing a bug with max pages counter 2025-09-16 01:47:31 +03:00
Karim shoair b7327466cf build: pump version up 2025-09-15 23:35:08 +03:00
Karim shoair 05c2d9aa00 feat(CLI): Make the session classes available in the shell 2025-09-15 23:34:36 +03:00
Karim shoair 5bcd4c2927 fix: A workaround for Playwright issue with page content retrieving on windows 2025-09-15 21:34:20 +03:00
Karim shoair 1547366c85 fix: fixing body issue with cli view command 2025-09-15 21:32:57 +03:00
Karim shoair a6aabda67c build: Pump version up 2025-09-15 13:52:09 +03:00
Karim shoair 7a7097b775 fix: Stop removing browser default page
Looks like it's problematic with some systems so it doesn't worth the trouble
2025-09-15 13:43:47 +03:00
Karim shoair f848e6b7f2 fix: Fixes to handle the new extra deps group 2025-09-15 00:19:19 +03:00
Karim shoair fdac239fcc build: Move click library to be part of the fetchers extra 2025-09-14 20:32:46 +03:00
Karim shoair ce4fc31f2c feat: Make .body return the passed content as it is without any processing
This makes it possible to download files and deal with non-HTML requests (ex: #81 )
2025-09-14 20:21:45 +03:00
Karim shoair 67ca139ff9 fix: Fixes for multiple encoding issues (#80 & #81 ) 2025-09-14 20:20:29 +03:00
Karim shoair 0b3509a295 fix(shell): Fix a small typo 2025-09-14 05:05:55 +03:00
Karim shoair 13d7e70cb7 refactor: Make all fetchers as an optional dependency group
+ Removing some dead code
2025-09-13 16:07:48 +03:00
Karim shoair 8c4d6158d6 feat(browser fetchers): A new option to control page JS 2025-09-13 03:59:12 +03:00
Karim shoair 330d03559c style: applying the new ruff rules to all files 2025-09-13 03:22:53 +03:00