Commit Graph

182 Commits

Author SHA1 Message Date
Karim shoair f848e6b7f2 fix: Fixes to handle the new extra deps group 2025-09-15 00:19:19 +03:00
Karim shoair 67ca139ff9 fix: Fixes for multiple encoding issues (#80 & #81 ) 2025-09-14 20:20:29 +03:00
Karim shoair 13d7e70cb7 refactor: Make all fetchers as an optional dependency group
+ Removing some dead code
2025-09-13 16:07:48 +03:00
Karim shoair 8c4d6158d6 feat(browser fetchers): A new option to control page JS 2025-09-13 03:59:12 +03:00
Karim shoair 330d03559c style: applying the new ruff rules to all files 2025-09-13 03:22:53 +03:00
Karim shoair 60be9dc816 feat: Validate all fetch-level parameters
So it's validated like the session-level ones. Needs improvement, but that's for later
2025-09-13 03:22:08 +03:00
Karim shoair 0c7db3c9d9 fix: validation keywords 2025-09-13 02:07:04 +03:00
Karim shoair adb82008e5 style: Using keywords for validation
Less lines of code and more stylish
2025-09-12 04:33:00 +03:00
Karim shoair 05a714980d docs: Update all docstring according to the new changes 2025-09-11 04:13:36 +03:00
Karim shoair 9838d741d0 refactor/feat(browser fetchers): make it possible to have a configuration per page in sessions
- Also, no need for the `page_action` argument function to return the page again
2025-09-11 03:48:55 +03:00
Karim shoair ecaec65049 refactor/fix(browser fetchers): Solving a logic bug in page rotation
- Pages that get reused in rotation are possibly contaminated from previous settings used on them, and some are stubborn to remove, so the new approach replaces finished pages with new ones.
- Removed the `max_pages` from sync `StealthySession` to match `DynamicSession` (Doesn't mean anything in sync code)
2025-09-08 16:08:57 +03:00
Karim shoair f298b76b8e perf(StealthySession): Close persistent context default page 2025-09-07 18:48:51 +03:00
Karim shoair 710f994339 perf(DynamicSession): Close persistent context default page 2025-09-07 18:48:40 +03:00
Karim shoair 4341477a66 perf(StealthyFetcher): Slight optimization 2025-09-07 04:58:55 +03:00
Karim shoair 72e26c23e8 fix(DynamicFetcher): Improve stealth mode 2025-09-07 03:13:51 +03:00
Karim shoair 6ad1bab2e4 feat(DynamicFetcher): add the ability to add init_script to context (#56) 2025-09-02 16:25:13 +03:00
Karim shoair 9543f171aa feat(StealthyFetcher): add the ability to add init_script to context 2025-09-02 16:19:22 +03:00
Karim shoair 967c0956a1 feat(DynamicSession): Improve stealth mode 2025-08-31 02:57:12 +03:00
Karim shoair 6eb580a6d0 feat(DynamicSession): Improve normal fetches speed 2025-08-31 02:56:34 +03:00
Karim shoair b5be2547d7 fix(dynamicSession): issue with headless 2025-08-31 01:29:47 +03:00
Karim shoair 2f402f4835 style: add flags for tests coverage
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
Karim shoair 009bbb20c0 refactor(dynamic Fetcher): small optimization 2025-08-16 23:20:54 +03:00
Karim shoair 6c9a960a44 fix(dynamicFetcher): Fix a bug with real_chrome argument 2025-08-16 16:01:32 +03:00
Karim shoair 825896c6a8 fix: improve checking for valid proxy and valid CDP URL 2025-08-03 16:31:16 +03:00
Karim shoair 5bb1266fa5 style: using isinstance function as the main way for type checking 2025-07-30 03:14:23 +03:00
Karim shoair ae9ccaec79 style: A lot of type hints correction
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
Karim shoair e7cdd39695 style: replacing os with Pathlib and small optimizations 2025-07-30 01:15:28 +03:00
Karim shoair 8ca940d768 style: removing dead code 2025-07-29 23:43:31 +03:00
Karim shoair 9bcb9e9d93 style: General type hints fixes and imports optimizing 2025-07-29 23:43:06 +03:00
Karim shoair b9c7a5af2e refactor: replace's Selector inpt (text/body) with 1 argument called content 2025-07-29 06:08:15 +03:00
Karim shoair 264ae02aa7 refactor: huge change, many features/class got a better naming
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
Karim shoair d1aa0be6e4 refactor(StealthyFetcher): Remove virtual mode and use persistent context
Solves #64 completely too
2025-07-27 03:41:28 +03:00
Karim shoair ecef751468 fix: small fix for fetcher 2025-07-14 02:14:54 +03:00
Karim shoair 63a9db21a1 refactor(DynamicSession): Optimization + Removing max_pages from sync version 2025-07-06 04:59:22 +03:00
Karim shoair 6d79927233 refactor(Fetcher): Fix the issue of caching impersonation state + Less code duplication
It now creates a session with each request, but at the same time, it's still faster than the fetcher in v0.2.99 by 20% with more features enabled.
2025-07-06 02:21:15 +03:00
Karim shoair dbf7aa7761 feat(DynamicFetcher): Use persistent context by default for better stealth
And might solve the sandbox issue with GitHub
2025-06-27 03:02:46 +03:00
Karim shoair 8395db9121 fix(controllers): Fix issue with async adding stealth scripts 2025-06-27 01:39:32 +03:00
Karim shoair 37e842b796 fix(controllers): Fix issue with adding stealth scripts 2025-06-27 01:04:58 +03:00
Karim shoair 53318f3a2a fix(StealthyFetcher): Fix passed proxy type 2025-06-26 16:36:30 +03:00
Karim shoair 8d44e9c51c fix(proxy): fix proxy unpacking 2025-06-26 15:49:53 +03:00
Karim shoair 38851698ad refactor: optimize imports and docstrings correction 2025-06-25 23:15:32 +03:00
Karim shoair 724e3a648e fix(FetcherSession): Use Sentinel pattern to solve arguments precedence issue 2025-06-25 03:53:54 +03:00
Karim shoair 63ff12bcd6 docs: correcting a small mistake 2025-06-23 03:38:34 +03:00
Karim shoair 78b81c537f fix(fetchers): Fix the bug of referer and google_search argument conflict 2025-06-23 03:36:11 +03:00
Karim shoair 69b983b42f feat(fetchers): Improve StealthyFetcher + Adding StealthySession/AsyncStealthySession classes 2025-06-23 03:23:28 +03:00
Karim shoair 3754547e58 refactor: removing unused code 2025-06-23 03:20:49 +03:00
Karim shoair 460b4443c2 refactor(controllers): Improve type validation 2025-06-22 19:59:40 +03:00
Karim shoair 0fe04e499f refactor(validators): Optimize imports 2025-06-22 18:55:13 +03:00
Karim shoair fc42487a5f refactor(controllers): Optimize imports 2025-06-22 18:53:44 +03:00
Karim shoair 9c44ad1930 fix(AsyncFetcher): Fix stealthy_headers issue 2025-06-22 03:54:36 +03:00