freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
517 Commits 1 Branch 0 Tags
403f7bfe72412daae9fda714df2cbc730ee76ef9
Commit Graph

517 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Karim shoair e9e606bca5 feat(StealthyFetcher): Disable Cross-Origin-Opener-Policy by default for async_fetch 
This allows elements in cross-origin iframes, such as the Turnstile checkbox, to be clicked.
.
Forgot to add it with the other commit
 2025-01-28 23:53:12 +02:00
Karim shoair 675c0110e6 feat(StealthyFetcher): Make ads enabled by default for performance boost 2025-01-28 23:51:31 +02:00
Karim shoair 52b66c98ec feat(StealthyFetcher): Disable Cross-Origin-Opener-Policy by default 
This allows elements in cross-origin iframes, such as the Turnstile checkbox, to be clicked.
 2025-01-28 23:47:47 +02:00
Karim shoair 1332be7e63 fix(autocompletion): Fix the defaults import 
It runs autocompletion on some IDEs
 2025-01-28 23:44:58 +02:00
Karim shoair c2447e069e build: Pumping version up 2025-01-28 23:41:48 +02:00
Karim shoair 99fe842bc9 build: updating dependencies 2025-01-28 23:40:56 +02:00
Karim shoair 21291958f1 docs(README file): Better more-friendly description 2025-01-28 23:10:56 +02:00
Karim shoair 198b76f5d5 Test: disable stealth parameter 
It causes issues with Github Actions
 2025-01-28 13:43:36 +02:00
Karim shoair 7776e191e8 Update test_playwright.py 2025-01-28 13:42:11 +02:00
Karim shoair 75a965e802 Update FUNDING.yml 2025-01-27 21:14:50 +02:00
Karim shoair 0e1dcd4f29 Adding Scrapeless sponsor 2025-01-06 11:59:19 +02:00
Karim shoair 32d96606ea Merge pull request #27 from D4Vinci/dev 
v0.2.92
 2024-12-26 20:03:24 +02:00
Karim shoair c5d83a11ad build: Pumping version up 2024-12-26 19:57:25 +02:00
Karim shoair ef284234e6 docs: update README file to reflect new installation instructions 2024-12-26 19:56:37 +02:00
Karim shoair 03b52aa74c feat(cli): adding terminal command scrapling install 
Instead of making users install browsers by themself
 2024-12-26 19:46:22 +02:00
Karim shoair 2006be2173 fix(StealthyFetcher): Use more dependable response data 2024-12-26 00:29:42 +02:00
Karim shoair f9b85cfbac fix(PlaywrightFetcher): Use more dependable response data 2024-12-26 00:27:13 +02:00
Karim shoair 3cca3fd56b style(parser): more accurate type hints 2024-12-21 16:17:02 +02:00
Karim shoair ee599141fe Merge pull request #26 from D4Vinci/dev 
v0.2.91
 2024-12-19 13:51:53 +02:00
Karim shoair bb398693b8 feat(Fetcher): Adding support to proxies using the SOCKS protocol 2024-12-19 13:30:43 +02:00
Karim shoair f74db648bc build: Pumping up the version to 0.2.91 2024-12-19 13:29:45 +02:00
Karim shoair 5be8c2a95e style(Fetchers): Setting correct type hint for wait_selector_state argument 2024-12-19 13:22:12 +02:00
Karim shoair 94745b1555 fix(StealthyFetcher): Wait for the real response status code 
This will return the real status response after redirection for websites that have Cloudflare instead of Playwright API's default behavior.
 2024-12-19 13:08:41 +02:00
Karim shoair 844cb0c6a2 fix(PlaywrightFetcher): Wait for the real response status code 
This will return the real status response after redirection for websites that have Cloudflare instead of Playwright's default behavior.
 2024-12-19 13:00:19 +02:00
Karim shoair 6652b8ac18 fix: solve the response log issue so now you get log for every response from Fetchers 2024-12-19 12:29:34 +02:00
Karim shoair 60df72cb23 Merge pull request #25 from D4Vinci/dev 
v0.2.9
 2024-12-16 14:42:13 +02:00
Karim shoair 838b8b1196 style: fixing a typo in logging 2024-12-16 13:39:30 +02:00
Karim shoair c282cb3cb7 fix: Stopped the log spamming that happens with multiple instances creation 2024-12-16 13:35:48 +02:00
Karim shoair ac95600650 docs: add an example that uses page.urljoin 2024-12-16 13:27:14 +02:00
Karim shoair 20ef45349a test: Rewrite automatch tests to a cleaner version and adding async test 2024-12-16 13:18:19 +02:00
Karim shoair 69e31612a5 test: Rewrite parser tests to a cleaner version and adding more tests 2024-12-16 13:17:09 +02:00
Karim shoair 3ff0d55c3e chore: Delete former-sponsor banner 2024-12-16 12:47:43 +02:00
Karim shoair 43b41b3733 docs: fix TOC header 2024-12-16 12:41:07 +02:00
Karim shoair 6cf5ce9719 docs: Adding async examples and fixing some typos 2024-12-16 12:39:44 +02:00
Karim shoair 05c6eeb387 fix: add AsyncFetcher to top-level shortcuts 2024-12-16 12:37:29 +02:00
Karim shoair d66ebc19a5 fix(Fetchers/disable_resources): Fixing the logic for intercepting resources 2024-12-16 12:35:25 +02:00
Karim shoair eceef48ef5 fix(Fetchers/page_action): Fixing logic 2024-12-16 12:34:23 +02:00
Karim shoair 3ecffcb81c style: Rewrite sync Fetchers tests to a cleaner version 2024-12-16 01:27:18 +02:00
Karim shoair 2606f7a12d test: clearer naming for async tests 2024-12-16 01:01:59 +02:00
Karim shoair 448587ff41 test: add tests for StealthyFetcher async fetch 2024-12-16 01:00:05 +02:00
Karim shoair 79a911e24e feat(StealthyFetcher): Add async fetch support 2024-12-16 00:59:19 +02:00
Karim shoair efb3270c67 style: Rewrite asyncFetcher tests to a cleaner version 2024-12-16 00:47:02 +02:00
Karim shoair 361ee440aa test: adding pytest-asyncio plugin to tests requirements file 2024-12-16 00:20:26 +02:00
Karim shoair de015f2b18 chore: Restructuring fetchers into cleaner structure 2024-12-16 00:19:48 +02:00
Karim shoair aac77e488c test: add test for PlaywrightFetcher Async support 2024-12-16 00:17:13 +02:00
Karim shoair af4f2c0f74 feat(PlaywrightFetcher): Add async support for PlaywrightFetcher 2024-12-16 00:15:47 +02:00
Karim shoair 889c111855 refactor(Playwright Engine): Separate what we can for cleaner code and the async function later 
The caching will give a slight performance increase with bulk requests
 2024-12-15 23:25:02 +02:00
Karim shoair 6c17bd886e style: using better data structures for constants 
This will cause a slight performance increase
 2024-12-15 23:19:47 +02:00
Karim shoair a90192df03 build: adding tests for AsyncFetcher class 2024-12-15 16:54:52 +02:00
Karim shoair b10cfd3643 feat: Adding AsyncFetcher class version of Fetcher 
The first step in fully supporting async
 2024-12-15 16:51:58 +02:00