Commit Graph

232 Commits

Author SHA1 Message Date
Karim shoair 8a272cf19b feat/refactor(fetchers): Replacing PlayWrightFetcher with DynamicFetcher and adding session classes 2025-06-22 02:10:45 +03:00
Karim shoair 0a570a7ca7 docs: Updating doc strings 2025-06-22 01:59:38 +03:00
Karim shoair d049f63404 fix(fetcher): Fix impersonate and headers generation conflict 2025-06-20 03:29:05 +03:00
Karim shoair 9bfc900140 feat(navigation tools): Improvements 2025-06-19 04:07:45 +03:00
Karim shoair 145b710960 feat(fetchers): Adding the foundation of the new browser-based fetchers logic 2025-06-19 04:06:52 +03:00
Karim shoair 2bdee60004 fix(fingerprints): Fixing headers generation for requests 2025-06-15 03:37:22 +03:00
Karim shoair 1f34ac22f6 style(fetcher): Move the default UA line 2025-06-15 03:21:21 +03:00
Karim shoair 7efbcd33c3 refactor(fingerprints): Remove unused function and improve another 2025-06-15 03:16:27 +03:00
Karim shoair 6e837f6a12 refactor(StealthyFetcher): Improve OS name extraction 2025-06-14 18:57:31 +03:00
Karim shoair 178eed7533 feat/fix(Fetcher): Adding http3 support + Fix stealthy_headers
And some fixes here and there to the docs
2025-05-31 03:35:46 +03:00
Karim shoair c46ca8873f refactor(fetchers): Optimizing fetchers + making PlayWrightFetcher 10% faster
Check out the Discord server for full details
2025-05-25 21:14:24 +03:00
Karim shoair 9e68e60161 feat(Fetcher): Replacing httpx + Adding FetcherSession
Check out the discord server for details
2025-05-25 21:10:20 +03:00
Karim shoair bf72678480 feat(cookies): The ability to pass cookies to browser fetchers 2025-05-10 19:36:53 +03:00
Karim shoair c84143129e fix(fetchers): Adjusting all cookies returned from each fetcher
The cookies returned now can be passed again to each fetcher's engine without further validation by us.
2025-05-10 19:16:34 +03:00
Karim shoair bd47c070aa fix(StealthyFetcher): Adjustments 2025-05-10 17:49:03 +03:00
Karim shoair fd33e35ff8 feat(StealthyFetcher): adding the option to solve Cloudflare Turnstile 2025-05-08 01:08:04 +03:00
Karim shoair 6c0b786dea docs(playwrightfetcher): improving docstring 2025-05-08 01:04:08 +03:00
Karim shoair 1240ee3eb0 fix(Fetcher): Fix the issue of passing cookies 2025-04-29 03:01:19 +03:00
Karim shoair 0c8dd63f87 chore: migrating to ruff and updating pre-commit hooks 2025-04-13 17:32:00 +02:00
Karim shoair a79bf3cc9d feat(StealthyFetcher): adding the wait option
This will make fetcher sleep after fetch before closing the page
2025-04-08 03:08:48 +02:00
Karim shoair 8f5d66f678 fix(StealthyFetcher): fix redirection issue with async_fetch 2025-04-08 02:58:10 +02:00
Karim shoair d6a3434402 fix(PlayWrightFetcher): fix redirection issue with async_fetch 2025-04-08 02:57:01 +02:00
Karim shoair 128f7b9d6f feat(PlayWrightFetcher): adding the option to sleep after fetch before closing the page 2025-04-08 02:54:46 +02:00
Karim shoair eb4ca2c6a4 fix(StealthyFetcher): change additional_arguments to a dictionary argument 2025-04-08 02:24:43 +02:00
Karim shoair 811cd2d9b0 refactor: make auto_match disabled by default 2025-03-26 19:25:19 +02:00
Karim shoair fec8215631 docs: Clearer docs for recent changes 2025-03-26 05:56:49 +02:00
Karim shoair d33e738c53 feat(StealthyFetcher): The ability to pass additional arguments to Camoufox 2025-03-26 05:36:05 +02:00
Karim shoair 7c0c23fd33 feat(unified import): Make one import for all fetchers
Why I didn't think of that from the start? IDK
2025-03-26 02:48:34 +02:00
Karim shoair ff3962b1ad fix: Adjusting cache size in all the library 2025-03-17 06:46:41 +02:00
Karim shoair 7bef44ea4c fix: Limit caching in all the library
To lower memory usage with high numbers of requests. Possibly improve case #48
2025-03-15 02:01:16 +02:00
Karim shoair b5c7a8c36d refactor/fix(Fetcher engine): Improvements and fixes
- Added the retries argument to the async methods, I don't understand how I missed it but that's why the code is rewritten now for easier maintenance.
2025-03-15 01:46:38 +02:00
Karim shoair 0c31fb7c0e refactor/fix(Playwright engine): Improvements and fixes 2025-03-15 01:45:12 +02:00
Karim shoair a2123fe91c refactor/fix(Camo engine): Improvements and fixes 2025-03-15 01:44:49 +02:00
Karim shoair 5f4ec3ba3a fix(Fetcher): make stealthy_headers don't overwrite user headers
Fix for #39
2025-02-25 23:14:59 +02:00
Karim shoair aa4d365901 fix(StealthyFetcher): no need to disable coop now
Camoufox can now click inside coop iframes without it
2025-02-22 18:46:02 +02:00
Karim shoair 10ecd41444 feat(StealthyFetcher): Add the redirections history 2025-02-22 18:45:17 +02:00
Karim shoair 8049ac0f28 feat(PlayWrightFetcher): Add the redirections history 2025-02-22 17:47:27 +02:00
Karim shoair 3d2418b67a feat(Fetcher): Add the redirections history to Fetcher
Feature requested in #32
2025-02-22 16:58:54 +02:00
Karim shoair ae60c2c0ad fix(playwright engine): bug in checking nstbrowser_config type 2025-01-30 20:07:19 +02:00
Karim shoair 85f98708b2 feat(StealthyFetcher): Use memory cache by default for performance boost
This is instead of the default disk-based cache.
2025-01-28 23:55:24 +02:00
Karim shoair e9e606bca5 feat(StealthyFetcher): Disable Cross-Origin-Opener-Policy by default for async_fetch
This allows elements in cross-origin iframes, such as the Turnstile checkbox, to be clicked.
.
Forgot to add it with the other commit
2025-01-28 23:53:12 +02:00
Karim shoair 675c0110e6 feat(StealthyFetcher): Make ads enabled by default for performance boost 2025-01-28 23:51:31 +02:00
Karim shoair 52b66c98ec feat(StealthyFetcher): Disable Cross-Origin-Opener-Policy by default
This allows elements in cross-origin iframes, such as the Turnstile checkbox, to be clicked.
2025-01-28 23:47:47 +02:00
Karim shoair 2006be2173 fix(StealthyFetcher): Use more dependable response data 2024-12-26 00:29:42 +02:00
Karim shoair f9b85cfbac fix(PlaywrightFetcher): Use more dependable response data 2024-12-26 00:27:13 +02:00
Karim shoair 5be8c2a95e style(Fetchers): Setting correct type hint for wait_selector_state argument 2024-12-19 13:22:12 +02:00
Karim shoair 94745b1555 fix(StealthyFetcher): Wait for the real response status code
This will return the real status response after redirection for websites that have Cloudflare instead of Playwright API's default behavior.
2024-12-19 13:08:41 +02:00
Karim shoair 844cb0c6a2 fix(PlaywrightFetcher): Wait for the real response status code
This will return the real status response after redirection for websites that have Cloudflare instead of Playwright's default behavior.
2024-12-19 13:00:19 +02:00
Karim shoair 6652b8ac18 fix: solve the response log issue so now you get log for every response from Fetchers 2024-12-19 12:29:34 +02:00
Karim shoair c282cb3cb7 fix: Stopped the log spamming that happens with multiple instances creation 2024-12-16 13:35:48 +02:00