 Karim shoair
|
460b4443c2
|
refactor(controllers): Improve type validation
|
2025-06-22 19:59:40 +03:00 |
|
|
Karim shoair
|
0fe04e499f
|
refactor(validators): Optimize imports
|
2025-06-22 18:55:13 +03:00 |
|
|
Karim shoair
|
fc42487a5f
|
refactor(controllers): Optimize imports
|
2025-06-22 18:53:44 +03:00 |
|
|
Karim shoair
|
4769a43525
|
test: Fix for Github CI
|
2025-06-22 04:02:19 +03:00 |
|
|
Karim shoair
|
9c44ad1930
|
fix(AsyncFetcher): Fix stealthy_headers issue
|
2025-06-22 03:54:36 +03:00 |
|
|
Karim shoair
|
8100a2a865
|
test: fix for Github CI
|
2025-06-22 03:07:46 +03:00 |
|
|
Karim shoair
|
8b46e3a3a5
|
tests: Updating tests according to the new changes
|
2025-06-22 02:12:39 +03:00 |
|
|
Karim shoair
|
f28b4d3653
|
docs: Updating Readme accordingly to the new naming
|
2025-06-22 02:12:03 +03:00 |
|
|
Karim shoair
|
b77c5b7abf
|
refactor: updating top level shortcuts
|
2025-06-22 02:11:26 +03:00 |
|
|
Karim shoair
|
8a272cf19b
|
feat/refactor(fetchers): Replacing PlayWrightFetcher with DynamicFetcher and adding session classes
|
2025-06-22 02:10:45 +03:00 |
|
|
Karim shoair
|
1e51fc9738
|
refactor: Removing defaults as said
|
2025-06-22 02:07:34 +03:00 |
|
|
Karim shoair
|
0a570a7ca7
|
docs: Updating doc strings
|
2025-06-22 01:59:38 +03:00 |
|
|
Karim shoair
|
d049f63404
|
fix(fetcher): Fix impersonate and headers generation conflict
|
2025-06-20 03:29:05 +03:00 |
|
|
Karim shoair
|
194ce24201
|
build: update deps
|
2025-06-19 15:19:23 +03:00 |
|
|
Karim shoair
|
9bfc900140
|
feat(navigation tools): Improvements
|
2025-06-19 04:07:45 +03:00 |
|
|
Karim shoair
|
145b710960
|
feat(fetchers): Adding the foundation of the new browser-based fetchers logic
|
2025-06-19 04:06:52 +03:00 |
|
|
Karim shoair
|
23b883f05a
|
build: Add msgspec to the deps
It will be used very soon
|
2025-06-19 04:04:25 +03:00 |
|
|
Karim shoair
|
3e63fa2523
|
fix(shell): Hide the automatic tips and solve the namespace issue
|
2025-06-19 04:03:13 +03:00 |
|
|
Karim shoair
|
2bdee60004
|
fix(fingerprints): Fixing headers generation for requests
|
2025-06-15 03:37:22 +03:00 |
|
|
Karim shoair
|
1f34ac22f6
|
style(fetcher): Move the default UA line
|
2025-06-15 03:21:21 +03:00 |
|
|
Karim shoair
|
7efbcd33c3
|
refactor(fingerprints): Remove unused function and improve another
|
2025-06-15 03:16:27 +03:00 |
|
|
Karim shoair
|
fcac10fa6d
|
build: update deps
|
2025-06-14 22:05:57 +03:00 |
|
|
Karim shoair
|
6e837f6a12
|
refactor(StealthyFetcher): Improve OS name extraction
|
2025-06-14 18:57:31 +03:00 |
|
|
Karim shoair
|
3068bb1356
|
build: update deps
|
2025-06-14 18:56:31 +03:00 |
|
|
Karim shoair
|
b55d97500b
|
build: Updating playwright deps
|
2025-06-01 00:06:08 +03:00 |
|
|
Karim shoair
|
bbdac4d967
|
build(setup): Replacing old setup style with new TOML format
|
2025-05-31 23:24:41 +03:00 |
|
|
Karim shoair
|
178eed7533
|
feat/fix(Fetcher): Adding http3 support + Fix stealthy_headers
And some fixes here and there to the docs
|
2025-05-31 03:35:46 +03:00 |
|
|
Karim shoair
|
af643e4c01
|
test: Updating old httpx tests for curl_cffi
|
2025-05-25 21:18:03 +03:00 |
|
|
Karim shoair
|
c46ca8873f
|
refactor(fetchers): Optimizing fetchers + making PlayWrightFetcher 10% faster
Check out the Discord server for full details
|
2025-05-25 21:14:24 +03:00 |
|
|
Karim shoair
|
9e68e60161
|
feat(Fetcher): Replacing httpx + Adding FetcherSession
Check out the discord server for details
|
2025-05-25 21:10:20 +03:00 |
|
|
Karim shoair
|
5c420b7384
|
Merge branch 'main' into dev
|
2025-05-23 20:29:05 +03:00 |
|
|
Karim shoair
|
561c8b0fcf
|
Chore: adding Swiftproxy ad
|
2025-05-23 14:28:02 +03:00 |
|
|
Karim shoair
|
f28e9c2b1e
|
Chore: adding Swiftproxy image
|
2025-05-23 14:19:08 +03:00 |
|
|
Karim shoair
|
f105518fd9
|
Merge branch 'main' into dev
|
2025-05-18 13:55:29 +03:00 |
|
|
Karim shoair
|
e7f8e5794a
|
fix(tests): fix for GitHub actions
|
2025-05-13 15:49:52 +03:00 |
|
|
Karim shoair
|
a941c02e1c
|
Update README.md
|
2025-05-13 15:42:02 +03:00 |
|
|
Karim shoair
|
43e3642719
|
Update Scrapeless segment
|
2025-05-13 15:40:49 +03:00 |
|
|
Karim shoair
|
fc535fa208
|
perf: drop w3lib dependency
|
2025-05-11 20:22:15 +03:00 |
|
|
Karim shoair
|
3268617240
|
test(cookies): Adjust cookies unit tests to the new changes
|
2025-05-10 20:08:31 +03:00 |
|
|
Karim shoair
|
bf72678480
|
feat(cookies): The ability to pass cookies to browser fetchers
|
2025-05-10 19:36:53 +03:00 |
|
|
Karim shoair
|
c84143129e
|
fix(fetchers): Adjusting all cookies returned from each fetcher
The cookies returned now can be passed again to each fetcher's engine without further validation by us.
|
2025-05-10 19:16:34 +03:00 |
|
|
Karim shoair
|
bd47c070aa
|
fix(StealthyFetcher): Adjustments
|
2025-05-10 17:49:03 +03:00 |
|
|
Karim shoair
|
cd3321a6f0
|
fix(tests): fix for GitHub actions
|
2025-05-08 03:11:01 +03:00 |
|
|
Karim shoair
|
fd33e35ff8
|
feat(StealthyFetcher): adding the option to solve Cloudflare Turnstile
|
2025-05-08 01:08:04 +03:00 |
|
|
Karim shoair
|
6c0b786dea
|
docs(playwrightfetcher): improving docstring
|
2025-05-08 01:04:08 +03:00 |
|
|
Karim shoair
|
74fa1bfbed
|
feat(shell): Add support to curl -b argument
|
2025-04-30 04:23:05 +03:00 |
|
|
Karim shoair
|
8f3b2092c2
|
fix(cli): Update page object after curl request
|
2025-04-29 03:14:41 +03:00 |
|
|
Karim shoair
|
1240ee3eb0
|
fix(Fetcher): Fix the issue of passing cookies
|
2025-04-29 03:01:19 +03:00 |
|
|
Karim shoair
|
98ad50d5aa
|
feat(cli): Adding two new commands (uncurl/curl2fetcher)
|
2025-04-29 02:45:50 +03:00 |
|
|
Karim shoair
|
a9dded34f4
|
fix(install): Fix error with spaces in Python's path (#57)
|
2025-04-26 04:10:38 +03:00 |
|
|
Karim shoair
|
e4bedc779b
|
build: Bumping version up for beta testers
|
2025-04-22 05:17:08 +02:00 |
|
|
Karim shoair
|
49b7ae13f6
|
feat(cli): Adding Scrapling Shell feature
|
2025-04-22 05:15:57 +02:00 |
|
|
Karim shoair
|
5ac5d60d77
|
Merge branch 'main' into dev
|
2025-04-18 01:09:54 +02:00 |
|
|
Karim shoair
|
34d9d02636
|
chore: update Evomi banner
|
2025-04-15 14:24:22 +02:00 |
|
|
Karim shoair
|
d5592662b8
|
chore: add Evomi banner
|
2025-04-15 14:20:49 +02:00 |
|
|
Karim shoair
|
d9f28b2ba2
|
chore: updating discord link
|
2025-04-15 03:27:50 +02:00 |
|
|
Karim shoair
|
4d9f7b27ff
|
docs: Update README.md
|
2025-04-15 03:23:34 +02:00 |
|
|
Karim shoair
|
ad83c03472
|
chore: Moving Evomi up
|
2025-04-15 00:44:16 +02:00 |
|
|
Karim shoair
|
1aacdd6009
|
chore: updating Scrapeless ad
|
2025-04-14 17:57:42 +02:00 |
|
|
Karim shoair
|
482a4a9308
|
Merge branch 'main' into dev
|
2025-04-13 17:34:32 +02:00 |
|
|
Karim shoair
|
784c4e5bb8
|
docs: update docs link in CONTRIBUTING file
|
2025-04-13 17:34:23 +02:00 |
|
|
Karim shoair
|
0c8dd63f87
|
chore: migrating to ruff and updating pre-commit hooks
|
2025-04-13 17:32:00 +02:00 |
|
|
Karim shoair
|
f34b42ea33
|
docs: add discord server to the issues tab
|
2025-04-13 15:22:13 +02:00 |
|
|
Karim shoair
|
f13cafefb8
|
docs: add the discord server
|
2025-04-13 03:56:42 +02:00 |
|
|
Karim shoair
|
ba42775d97
|
docs: Update README.md
|
2025-04-12 22:37:57 +02:00 |
|
|
Karim shoair
|
211825a49e
|
docs: update README logo
|
2025-04-12 22:36:25 +02:00 |
|
|
Karim shoair
|
ca9daf23de
|
docs: Adding new poster
|
2025-04-12 22:31:58 +02:00 |
|
|
Karim shoair
|
8b93857546
|
docs: removing old documentation
Checkout the website at https://scrapling.readthedocs.io
|
2025-04-08 06:56:06 +02:00 |
|
|
Karim shoair
|
96e3c3d2b5
|
Update publish.yml
|
2025-04-08 06:34:21 +02:00 |
|
|
Karim shoair
|
c3b141ed4b
|
Merge pull request #55 from D4Vinci/dev
v0.2.99
|
2025-04-08 06:27:07 +02:00 |
|
|
Karim shoair
|
cf81e4e1a9
|
docs: Update README file
|
2025-04-08 06:06:12 +02:00 |
|
|
Karim shoair
|
8833a0916e
|
build: update docs URL
|
2025-04-08 06:05:02 +02:00 |
|
|
Karim shoair
|
a79bf3cc9d
|
feat(StealthyFetcher): adding the wait option
This will make fetcher sleep after fetch before closing the page
|
2025-04-08 03:08:48 +02:00 |
|
|
Karim shoair
|
8f5d66f678
|
fix(StealthyFetcher): fix redirection issue with async_fetch
|
2025-04-08 02:58:10 +02:00 |
|
|
Karim shoair
|
d6a3434402
|
fix(PlayWrightFetcher): fix redirection issue with async_fetch
|
2025-04-08 02:57:01 +02:00 |
|
|
Karim shoair
|
128f7b9d6f
|
feat(PlayWrightFetcher): adding the option to sleep after fetch before closing the page
|
2025-04-08 02:54:46 +02:00 |
|
|
Karim shoair
|
eb4ca2c6a4
|
fix(StealthyFetcher): change additional_arguments to a dictionary argument
|
2025-04-08 02:24:43 +02:00 |
|
|
Karim shoair
|
c36baf7b4c
|
test: Update tests to use current configuration
|
2025-03-26 21:56:04 +02:00 |
|
|
Karim shoair
|
8551d7eeea
|
docs: Update README reflect current version accurately
|
2025-03-26 19:25:49 +02:00 |
|
|
Karim shoair
|
811cd2d9b0
|
refactor: make auto_match disabled by default
|
2025-03-26 19:25:19 +02:00 |
|
|
Karim shoair
|
fec8215631
|
docs: Clearer docs for recent changes
|
2025-03-26 05:56:49 +02:00 |
|
|
Karim shoair
|
7025a7910e
|
build: Pump up the version
|
2025-03-26 05:44:45 +02:00 |
|
|
Karim shoair
|
8a2ac94243
|
docs: Updating the benchmarks table with current numbers
All libraries are updated to the latest version
|
2025-03-26 05:41:03 +02:00 |
|
|
Karim shoair
|
d33e738c53
|
feat(StealthyFetcher): The ability to pass additional arguments to Camoufox
|
2025-03-26 05:36:05 +02:00 |
|
|
Karim shoair
|
a0952dca8f
|
docs: Updating the README to reflect the new changes
|
2025-03-26 05:28:28 +02:00 |
|
|
Karim shoair
|
7ca8aaf2df
|
feat(custom_config): The ability to pass custom parser config to requests
|
2025-03-26 03:56:56 +02:00 |
|
|
Karim shoair
|
d7ef555d99
|
docs: Updating the README to reflect the new changes
|
2025-03-26 03:01:42 +02:00 |
|
|
Karim shoair
|
416bd919ba
|
fix: backward-compatibility with the new import logic
|
2025-03-26 02:49:02 +02:00 |
|
|
Karim shoair
|
7c0c23fd33
|
feat(unified import): Make one import for all fetchers
Why I didn't think of that from the start? IDK
|
2025-03-26 02:48:34 +02:00 |
|
|
Karim shoair
|
c004e388c4
|
build: fix auto-release workflow
Escape special chars by using a cleaner approach
|
2025-03-17 15:33:07 +02:00 |
|
|
Karim shoair
|
e60d0cb4db
|
Merge pull request #51 from D4Vinci/dev
v0.2.98
|
2025-03-17 15:24:09 +02:00 |
|
|
Karim shoair
|
9b0515b1cd
|
fix(defaults): Lazy Loading fetchers to lower memory usage on importing
|
2025-03-17 06:56:34 +02:00 |
|
|
Karim shoair
|
94f16ca1af
|
build: Pumping up the version
|
2025-03-17 06:47:43 +02:00 |
|
|
Karim shoair
|
ff3962b1ad
|
fix: Adjusting cache size in all the library
|
2025-03-17 06:46:41 +02:00 |
|
|
Karim shoair
|
18d2d29fa8
|
docs: use a better import to reflect the new changes
|
2025-03-17 06:29:32 +02:00 |
|
|
Karim shoair
|
bbc05ca164
|
fix(importing): Lazy Loading submodules to lower memory usage on importing
Load what you use instead of all submodules together
|
2025-03-17 06:24:42 +02:00 |
|
|
Karim shoair
|
68ac9d4d05
|
fix(parser): Lowering memory with replacing body property with html_content
|
2025-03-17 05:58:40 +02:00 |
|
|
Karim shoair
|
e5a022ec6a
|
fix(parser): Improving CSS selecting memory usage by ~144%
and speed by ~2%
|
2025-03-16 22:33:46 +02:00 |
|
|
Karim shoair
|
ed89161cd9
|
build: fixing release workflow
|
2025-03-15 03:26:40 +02:00 |
|
|
Karim shoair
|
01bd9a3d28
|
Merge pull request #49 from D4Vinci/dev
v0.2.97
|
2025-03-15 03:19:35 +02:00 |
|