Commit Graph

165 Commits

Author SHA1 Message Date
Karim shoair 967c0956a1 feat(DynamicSession): Improve stealth mode 2025-08-31 02:57:12 +03:00
Karim shoair 6eb580a6d0 feat(DynamicSession): Improve normal fetches speed 2025-08-31 02:56:34 +03:00
Karim shoair b5be2547d7 fix(dynamicSession): issue with headless 2025-08-31 01:29:47 +03:00
Karim shoair 2f402f4835 style: add flags for tests coverage
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
Karim shoair 009bbb20c0 refactor(dynamic Fetcher): small optimization 2025-08-16 23:20:54 +03:00
Karim shoair 6c9a960a44 fix(dynamicFetcher): Fix a bug with real_chrome argument 2025-08-16 16:01:32 +03:00
Karim shoair 825896c6a8 fix: improve checking for valid proxy and valid CDP URL 2025-08-03 16:31:16 +03:00
Karim shoair 5bb1266fa5 style: using isinstance function as the main way for type checking 2025-07-30 03:14:23 +03:00
Karim shoair ae9ccaec79 style: A lot of type hints correction
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
Karim shoair e7cdd39695 style: replacing os with Pathlib and small optimizations 2025-07-30 01:15:28 +03:00
Karim shoair 8ca940d768 style: removing dead code 2025-07-29 23:43:31 +03:00
Karim shoair 9bcb9e9d93 style: General type hints fixes and imports optimizing 2025-07-29 23:43:06 +03:00
Karim shoair b9c7a5af2e refactor: replace's Selector inpt (text/body) with 1 argument called content 2025-07-29 06:08:15 +03:00
Karim shoair 264ae02aa7 refactor: huge change, many features/class got a better naming
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
Karim shoair d1aa0be6e4 refactor(StealthyFetcher): Remove virtual mode and use persistent context
Solves #64 completely too
2025-07-27 03:41:28 +03:00
Karim shoair ecef751468 fix: small fix for fetcher 2025-07-14 02:14:54 +03:00
Karim shoair 63a9db21a1 refactor(DynamicSession): Optimization + Removing max_pages from sync version 2025-07-06 04:59:22 +03:00
Karim shoair 6d79927233 refactor(Fetcher): Fix the issue of caching impersonation state + Less code duplication
It now creates a session with each request, but at the same time, it's still faster than the fetcher in v0.2.99 by 20% with more features enabled.
2025-07-06 02:21:15 +03:00
Karim shoair dbf7aa7761 feat(DynamicFetcher): Use persistent context by default for better stealth
And might solve the sandbox issue with GitHub
2025-06-27 03:02:46 +03:00
Karim shoair 8395db9121 fix(controllers): Fix issue with async adding stealth scripts 2025-06-27 01:39:32 +03:00
Karim shoair 37e842b796 fix(controllers): Fix issue with adding stealth scripts 2025-06-27 01:04:58 +03:00
Karim shoair 53318f3a2a fix(StealthyFetcher): Fix passed proxy type 2025-06-26 16:36:30 +03:00
Karim shoair 8d44e9c51c fix(proxy): fix proxy unpacking 2025-06-26 15:49:53 +03:00
Karim shoair 38851698ad refactor: optimize imports and docstrings correction 2025-06-25 23:15:32 +03:00
Karim shoair 724e3a648e fix(FetcherSession): Use Sentinel pattern to solve arguments precedence issue 2025-06-25 03:53:54 +03:00
Karim shoair 63ff12bcd6 docs: correcting a small mistake 2025-06-23 03:38:34 +03:00
Karim shoair 78b81c537f fix(fetchers): Fix the bug of referer and google_search argument conflict 2025-06-23 03:36:11 +03:00
Karim shoair 69b983b42f feat(fetchers): Improve StealthyFetcher + Adding StealthySession/AsyncStealthySession classes 2025-06-23 03:23:28 +03:00
Karim shoair 3754547e58 refactor: removing unused code 2025-06-23 03:20:49 +03:00
Karim shoair 460b4443c2 refactor(controllers): Improve type validation 2025-06-22 19:59:40 +03:00
Karim shoair 0fe04e499f refactor(validators): Optimize imports 2025-06-22 18:55:13 +03:00
Karim shoair fc42487a5f refactor(controllers): Optimize imports 2025-06-22 18:53:44 +03:00
Karim shoair 9c44ad1930 fix(AsyncFetcher): Fix stealthy_headers issue 2025-06-22 03:54:36 +03:00
Karim shoair 8a272cf19b feat/refactor(fetchers): Replacing PlayWrightFetcher with DynamicFetcher and adding session classes 2025-06-22 02:10:45 +03:00
Karim shoair 0a570a7ca7 docs: Updating doc strings 2025-06-22 01:59:38 +03:00
Karim shoair d049f63404 fix(fetcher): Fix impersonate and headers generation conflict 2025-06-20 03:29:05 +03:00
Karim shoair 9bfc900140 feat(navigation tools): Improvements 2025-06-19 04:07:45 +03:00
Karim shoair 145b710960 feat(fetchers): Adding the foundation of the new browser-based fetchers logic 2025-06-19 04:06:52 +03:00
Karim shoair 2bdee60004 fix(fingerprints): Fixing headers generation for requests 2025-06-15 03:37:22 +03:00
Karim shoair 1f34ac22f6 style(fetcher): Move the default UA line 2025-06-15 03:21:21 +03:00
Karim shoair 7efbcd33c3 refactor(fingerprints): Remove unused function and improve another 2025-06-15 03:16:27 +03:00
Karim shoair 6e837f6a12 refactor(StealthyFetcher): Improve OS name extraction 2025-06-14 18:57:31 +03:00
Karim shoair 178eed7533 feat/fix(Fetcher): Adding http3 support + Fix stealthy_headers
And some fixes here and there to the docs
2025-05-31 03:35:46 +03:00
Karim shoair c46ca8873f refactor(fetchers): Optimizing fetchers + making PlayWrightFetcher 10% faster
Check out the Discord server for full details
2025-05-25 21:14:24 +03:00
Karim shoair 9e68e60161 feat(Fetcher): Replacing httpx + Adding FetcherSession
Check out the discord server for details
2025-05-25 21:10:20 +03:00
Karim shoair bf72678480 feat(cookies): The ability to pass cookies to browser fetchers 2025-05-10 19:36:53 +03:00
Karim shoair c84143129e fix(fetchers): Adjusting all cookies returned from each fetcher
The cookies returned now can be passed again to each fetcher's engine without further validation by us.
2025-05-10 19:16:34 +03:00
Karim shoair bd47c070aa fix(StealthyFetcher): Adjustments 2025-05-10 17:49:03 +03:00
Karim shoair fd33e35ff8 feat(StealthyFetcher): adding the option to solve Cloudflare Turnstile 2025-05-08 01:08:04 +03:00
Karim shoair 6c0b786dea docs(playwrightfetcher): improving docstring 2025-05-08 01:04:08 +03:00