freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
430 Commits 1 Branch 0 Tags
74fa1bfbed70d7fbbd447be75d1b6e7ed484ec1e
Commit Graph

41 Commits

Author SHA1 Message Date
Karim shoair 1240ee3eb0 fix(Fetcher): Fix the issue of passing cookies 2025-04-29 03:01:19 +03:00
Karim shoair 0c8dd63f87 chore: migrating to ruff and updating pre-commit hooks 2025-04-13 17:32:00 +02:00
Karim shoair a79bf3cc9d feat(StealthyFetcher): adding the wait option 
This will make fetcher sleep after fetch before closing the page
 2025-04-08 03:08:48 +02:00
Karim shoair 128f7b9d6f feat(PlayWrightFetcher): adding the option to sleep after fetch before closing the page 2025-04-08 02:54:46 +02:00
Karim shoair eb4ca2c6a4 fix(StealthyFetcher): change additional_arguments to a dictionary argument 2025-04-08 02:24:43 +02:00
Karim shoair d33e738c53 feat(StealthyFetcher): The ability to pass additional arguments to Camoufox 2025-03-26 05:36:05 +02:00
Karim shoair 7ca8aaf2df feat(custom_config): The ability to pass custom parser config to requests 2025-03-26 03:56:56 +02:00
Karim shoair 7c0c23fd33 feat(unified import): Make one import for all fetchers 
Why I didn't think of that from the start? IDK
 2025-03-26 02:48:34 +02:00
Karim shoair 4f75827641 refactor(fetchers): Clearer more accurate type hints 2025-03-15 01:47:41 +02:00
Karim shoair c0d912d16a fix(asyncFetcher): A typo made async version of put do post request instead 
Awkward moment
 2025-01-30 19:53:13 +02:00
Karim shoair 675c0110e6 feat(StealthyFetcher): Make ads enabled by default for performance boost 2025-01-28 23:51:31 +02:00
Karim shoair 5be8c2a95e style(Fetchers): Setting correct type hint for wait_selector_state argument 2024-12-19 13:22:12 +02:00
Karim shoair 79a911e24e feat(StealthyFetcher): Add async fetch support 2024-12-16 00:59:19 +02:00
Karim shoair af4f2c0f74 feat(PlaywrightFetcher): Add async support for PlaywrightFetcher 2024-12-16 00:15:47 +02:00
Karim shoair b10cfd3643 feat: Adding AsyncFetcher class version of Fetcher 
The first step in fully supporting async
 2024-12-15 16:51:58 +02:00
Karim shoair faf728a794 style: moving repeated arguments from inside the functions to __init__ 
Might give a slight performance increase too
 2024-12-15 16:49:48 +02:00
Karim shoair 445af3c702 perf: Give repeated usage of Fetcher a slight performance increase 
By caching the `StaticEngine` class instance
 2024-12-15 16:13:08 +02:00
Karim shoair 299793af3f feat: adding the retries argument for all methods of Fetcher class 2024-12-15 15:59:15 +02:00
Karim shoair bfe906315e feat: adding geoip parameter to the StealthyFetcher 2024-12-12 12:43:49 +02:00
Karim shoair d0f18951ac fix: Enable WebGL by default 2024-12-04 23:38:30 +02:00
Karim shoair 1a17b2ce46 Sorting all library imports with isort 2024-11-30 14:02:06 +02:00
Karim shoair dbd674aa7c Adding proxy support for basic requests 2024-11-26 21:24:41 +02:00
Karim shoair d6a59774ea StealthyFetcher - The option to disable ads with ublock origin addon 2024-11-26 20:23:02 +02:00
Karim shoair 60d551d839 PlaywrightFetcher - Add the option to set locale of the browser 2024-11-26 17:39:03 +02:00
Karim shoair 12d638ac2e Updating the doc to reflect new changes 2024-11-24 15:20:18 +02:00
Karim shoair a8c576ad28 Adding the real_chrome option to the fetcher 2024-11-24 15:15:08 +02:00
Karim shoair 7f330efc52 Turn-off the hide_canvas option by default 2024-11-24 13:44:16 +02:00
Karim shoair f443d774c1 Fixing the doc-string to match Sphinx 2024-11-19 14:33:54 +02:00
Karim shoair 90e38af552 Correcting the return string for the Response object in all functions/methods 2024-11-15 16:44:47 +02:00
Karim shoair 5e8275c3eb Adding the option to randomize the OS fingerprints with the StealthyFetcher 2024-11-15 16:36:32 +02:00
Karim shoair 2abb70218f Adding the proxy support to browser-based Fetchers 2024-11-15 14:43:03 +02:00
Karim shoair bd38fdb347 Corrections and rephrasing 2024-11-11 16:40:46 +02:00
Karim shoair 03091beb8b Small clarification 2024-11-09 13:40:50 +02:00
Karim shoair fd7d787ab0 Add extra_headers argument to PW and isolate google referer feature from stealth mode as google_search 2024-11-09 00:54:11 +02:00
Karim shoair fd34cd9afa Adding arguments google_search and extra_headers to StealthyFetcher 2024-11-09 00:43:24 +02:00
Karim shoair 530b54d964 Clearer doc-string 2024-11-09 00:17:41 +02:00
Karim shoair a1fd5ddc7e Correcting headless argument type 2024-11-06 20:18:44 +02:00
Karim shoair a0a3e62075 Adding addons and humanize arguments to StealthyFetcher 2024-11-06 20:14:42 +02:00
Karim shoair 6497b1b559 Type correction 2024-11-06 02:20:33 +02:00
Karim shoair bd612bb0e2 Updating the disable_resources doc-string 2024-11-04 20:16:04 +02:00
Karim shoair b8afb390ae Renaming fetcher file to fetchers for clarity 2024-11-03 22:43:16 +02:00