9e68e60161
feat(Fetcher): Replacing httpx + Adding FetcherSession
...
Check out the discord server for details
2025-05-25 21:10:20 +03:00
c84143129e
fix(fetchers): Adjusting all cookies returned from each fetcher
...
The cookies returned now can be passed again to each fetcher's engine without further validation by us.
2025-05-10 19:16:34 +03:00
1240ee3eb0
fix(Fetcher): Fix the issue of passing cookies
2025-04-29 03:01:19 +03:00
0c8dd63f87
chore: migrating to ruff and updating pre-commit hooks
2025-04-13 17:32:00 +02:00
ff3962b1ad
fix: Adjusting cache size in all the library
2025-03-17 06:46:41 +02:00
7bef44ea4c
fix: Limit caching in all the library
...
To lower memory usage with high numbers of requests. Possibly improve case #48
2025-03-15 02:01:16 +02:00
b5c7a8c36d
refactor/fix(Fetcher engine): Improvements and fixes
...
- Added the retries argument to the async methods, I don't understand how I missed it but that's why the code is rewritten now for easier maintenance.
2025-03-15 01:46:38 +02:00
5f4ec3ba3a
fix(Fetcher): make stealthy_headers don't overwrite user headers
...
Fix for #39
2025-02-25 23:14:59 +02:00
3d2418b67a
feat(Fetcher): Add the redirections history to Fetcher
...
Feature requested in #32
2025-02-22 16:58:54 +02:00
b10cfd3643
feat: Adding AsyncFetcher class version of Fetcher
...
The first step in fully supporting async
2024-12-15 16:51:58 +02:00
faf728a794
style: moving repeated arguments from inside the functions to __init__
...
Might give a slight performance increase too
2024-12-15 16:49:48 +02:00
445af3c702
perf: Give repeated usage of Fetcher a slight performance increase
...
By caching the `StaticEngine` class instance
2024-12-15 16:13:08 +02:00
299793af3f
feat: adding the retries argument for all methods of Fetcher class
2024-12-15 15:59:15 +02:00
e254341974
fix: forgot to stage it with last commit
2024-12-11 21:43:36 +02:00
193827e27b
refactor(api)!: Unifying log under 1 logger and removing debug parameter
...
So now you control the logging and the debugging from the shell through the logger with the name 'scrapling'
2024-12-11 21:41:37 +02:00
1a17b2ce46
Sorting all library imports with isort
2024-11-30 14:02:06 +02:00
dbd674aa7c
Adding proxy support for basic requests
2024-11-26 21:24:41 +02:00
f443d774c1
Fixing the doc-string to match Sphinx
2024-11-19 14:33:54 +02:00
572df6b3b4
Fixing the way Response object handles sub items in some edge cases
...
+ Do it with the least impact on performance
2024-11-16 13:54:55 +02:00
90e38af552
Correcting the return string for the Response object in all functions/methods
2024-11-15 16:44:47 +02:00
ab7b0d45ad
StaticEngine - Fix the headers bug
2024-11-13 23:13:39 +02:00
b3418f7f50
Corrections and rephrasing
2024-11-11 16:40:35 +02:00
282b130864
Static Engine - Doc strings and type annotations stuff
2024-11-03 22:25:34 +02:00
afe53332b8
Convert the request headers to dictionary to unify response object shape across engines
2024-11-03 13:01:48 +02:00
145c03daff
Big structure changes (check commit description)
...
- Moved most of the parser functions/files to the core package.
- Converted tools file to a package and made separate files for similar functions.
- Now all fetcher engines return a Response object
- Instead of selecting an engine to use and passing config to it, we have separate fetcher classes so the user can choose what to use while importing.
- I added a new custom fetcher so the user can create and use an engine.
- More...
2024-11-03 01:04:02 +02:00
8f8d08b568
Adding browsing engines
2024-10-31 02:11:33 +03:00
Karim shoair