freedak/Scrapling
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
431 Commits 1 Branch 0 Tags
6c0b786dea3811b59508c5d354dd7ab26a6a8bfb
Commit Graph

229 Commits

Author SHA1 Message Date
Karim shoair 6c0b786dea docs(playwrightfetcher): improving docstring 2025-05-08 01:04:08 +03:00
Karim shoair 74fa1bfbed feat(shell): Add support to curl -b argument 2025-04-30 04:23:05 +03:00
Karim shoair 8f3b2092c2 fix(cli): Update page object after curl request 2025-04-29 03:14:41 +03:00
Karim shoair 1240ee3eb0 fix(Fetcher): Fix the issue of passing cookies 2025-04-29 03:01:19 +03:00
Karim shoair 98ad50d5aa feat(cli): Adding two new commands (uncurl/curl2fetcher) 2025-04-29 02:45:50 +03:00
Karim shoair a9dded34f4 fix(install): Fix error with spaces in Python's path (#57) 2025-04-26 04:10:38 +03:00
Karim shoair e4bedc779b build: Bumping version up for beta testers 2025-04-22 05:17:08 +02:00
Karim shoair 49b7ae13f6 feat(cli): Adding Scrapling Shell feature 2025-04-22 05:15:57 +02:00
Karim shoair 0c8dd63f87 chore: migrating to ruff and updating pre-commit hooks 2025-04-13 17:32:00 +02:00
Karim shoair a79bf3cc9d feat(StealthyFetcher): adding the wait option 
This will make fetcher sleep after fetch before closing the page
 2025-04-08 03:08:48 +02:00
Karim shoair 8f5d66f678 fix(StealthyFetcher): fix redirection issue with async_fetch 2025-04-08 02:58:10 +02:00
Karim shoair d6a3434402 fix(PlayWrightFetcher): fix redirection issue with async_fetch 2025-04-08 02:57:01 +02:00
Karim shoair 128f7b9d6f feat(PlayWrightFetcher): adding the option to sleep after fetch before closing the page 2025-04-08 02:54:46 +02:00
Karim shoair eb4ca2c6a4 fix(StealthyFetcher): change additional_arguments to a dictionary argument 2025-04-08 02:24:43 +02:00
Karim shoair 811cd2d9b0 refactor: make auto_match disabled by default 2025-03-26 19:25:19 +02:00
Karim shoair fec8215631 docs: Clearer docs for recent changes 2025-03-26 05:56:49 +02:00
Karim shoair 7025a7910e build: Pump up the version 2025-03-26 05:44:45 +02:00
Karim shoair d33e738c53 feat(StealthyFetcher): The ability to pass additional arguments to Camoufox 2025-03-26 05:36:05 +02:00
Karim shoair 7ca8aaf2df feat(custom_config): The ability to pass custom parser config to requests 2025-03-26 03:56:56 +02:00
Karim shoair 416bd919ba fix: backward-compatibility with the new import logic 2025-03-26 02:49:02 +02:00
Karim shoair 7c0c23fd33 feat(unified import): Make one import for all fetchers 
Why I didn't think of that from the start? IDK
 2025-03-26 02:48:34 +02:00
Karim shoair 9b0515b1cd fix(defaults): Lazy Loading fetchers to lower memory usage on importing 2025-03-17 06:56:34 +02:00
Karim shoair 94f16ca1af build: Pumping up the version 2025-03-17 06:47:43 +02:00
Karim shoair ff3962b1ad fix: Adjusting cache size in all the library 2025-03-17 06:46:41 +02:00
Karim shoair bbc05ca164 fix(importing): Lazy Loading submodules to lower memory usage on importing 
Load what you use instead of all submodules together
 2025-03-17 06:24:42 +02:00
Karim shoair 68ac9d4d05 fix(parser): Lowering memory with replacing body property with html_content 2025-03-17 05:58:40 +02:00
Karim shoair e5a022ec6a fix(parser): Improving CSS selecting memory usage by ~144% 
and speed by ~2%
 2025-03-16 22:33:46 +02:00
Karim shoair 257b740e1d build: Pumping up the version 2025-03-15 02:38:46 +02:00
Karim shoair 7bef44ea4c fix: Limit caching in all the library 
To lower memory usage with high numbers of requests. Possibly improve case #48
 2025-03-15 02:01:16 +02:00
Karim shoair 4f75827641 refactor(fetchers): Clearer more accurate type hints 2025-03-15 01:47:41 +02:00
Karim shoair b5c7a8c36d refactor/fix(Fetcher engine): Improvements and fixes 
- Added the retries argument to the async methods, I don't understand how I missed it but that's why the code is rewritten now for easier maintenance.
 2025-03-15 01:46:38 +02:00
Karim shoair 0c31fb7c0e refactor/fix(Playwright engine): Improvements and fixes 2025-03-15 01:45:12 +02:00
Karim shoair a2123fe91c refactor/fix(Camo engine): Improvements and fixes 2025-03-15 01:44:49 +02:00
Karim shoair 94909099a9 fix(Parser): Change default text content to empty string instead of None 
For compatibility with previous fix/commit
 2025-03-15 00:56:19 +02:00
Karim shoair bf6a3a9c20 fix(TextHandler): auto handling non-string type variables 
Fix for #45
 2025-03-08 01:42:58 +02:00
Karim shoair f8662bee29 feat(cli): Adding the -f flag to install command 2025-03-05 03:32:43 +02:00
Karim shoair 087a85d615 fix(cli): Fix issue where install drops user into Python Shell 2025-03-05 03:28:07 +02:00
Karim shoair 5f06a34a2f build: Pumping version up 2025-03-05 03:13:47 +02:00
Karim shoair e9344a7384 fix(TextHandler): Fixing an issue with slicing 
Slicing was returning `TextHandlers` instead of `TextHandler`. This solves #41 and corrects the type hint
 2025-03-05 03:11:46 +02:00
Karim shoair dc702d2d7c build: pumping version up 2025-02-25 23:57:31 +02:00
Karim shoair 5f4ec3ba3a fix(Fetcher): make stealthy_headers don't overwrite user headers 
Fix for #39
 2025-02-25 23:14:59 +02:00
Karim shoair aa4d365901 fix(StealthyFetcher): no need to disable coop now 
Camoufox can now click inside coop iframes without it
 2025-02-22 18:46:02 +02:00
Karim shoair 10ecd41444 feat(StealthyFetcher): Add the redirections history 2025-02-22 18:45:17 +02:00
Karim shoair 8049ac0f28 feat(PlayWrightFetcher): Add the redirections history 2025-02-22 17:47:27 +02:00
Karim shoair 3d2418b67a feat(Fetcher): Add the redirections history to Fetcher 
Feature requested in #32
 2025-02-22 16:58:54 +02:00
Karim shoair f0db07a31f build: pumping up deps and version to 0.2.94 2025-02-22 15:25:35 +02:00
Karim shoair 390bd51cb1 fix(TextHandler): correcting case_sensitive argument logic 
This is awkward, I can't believe how this slipped from me!
 2025-02-22 15:20:11 +02:00
Karim shoair 7c35341d85 refactor(parser/get_all_text): Cleaner and a bit faster implementation 2025-01-31 02:47:49 +02:00
Karim shoair a6fd25f64c fix(parser): fix traverse selectors 2025-01-31 01:57:19 +02:00
Karim shoair a8d475afb0 feat(parser): Make all returned html as TextHandler 
So you can do regex on raw html if wanted etc...
 2025-01-31 01:38:48 +02:00