debe03256b
refactor: Making all the codebase acceptable by PyRight
...
Also fixes #97
2025-10-05 04:03:39 +03:00
a9d05cc0ef
style: Removing dead code/docstrings and correcting type hints
2025-09-19 04:49:30 +03:00
13d7e70cb7
refactor: Make all fetchers as an optional dependency group
...
+ Removing some dead code
2025-09-13 16:07:48 +03:00
330d03559c
style: applying the new ruff rules to all files
2025-09-13 03:22:53 +03:00
825896c6a8
fix: improve checking for valid proxy and valid CDP URL
2025-08-03 16:31:16 +03:00
ae9ccaec79
style: A lot of type hints correction
...
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
e7cdd39695
style: replacing os with Pathlib and small optimizations
2025-07-30 01:15:28 +03:00
8d44e9c51c
fix(proxy): fix proxy unpacking
2025-06-26 15:49:53 +03:00
9bfc900140
feat(navigation tools): Improvements
2025-06-19 04:07:45 +03:00
0c8dd63f87
chore: migrating to ruff and updating pre-commit hooks
2025-04-13 17:32:00 +02:00
ff3962b1ad
fix: Adjusting cache size in all the library
2025-03-17 06:46:41 +02:00
7bef44ea4c
fix: Limit caching in all the library
...
To lower memory usage with high numbers of requests. Possibly improve case #48
2025-03-15 02:01:16 +02:00
d66ebc19a5
fix(Fetchers/disable_resources): Fixing the logic for intercepting resources
2024-12-16 12:35:25 +02:00
af4f2c0f74
feat(PlaywrightFetcher): Add async support for PlaywrightFetcher
2024-12-16 00:15:47 +02:00
193827e27b
refactor(api)!: Unifying log under 1 logger and removing debug parameter
...
So now you control the logging and the debugging from the shell through the logger with the name 'scrapling'
2024-12-11 21:41:37 +02:00
1a17b2ce46
Sorting all library imports with isort
2024-11-30 14:02:06 +02:00
97fecc9d5f
Style adjustments by flake8
2024-11-29 21:14:52 +02:00
2abb70218f
Adding the proxy support to browser-based Fetchers
2024-11-15 14:43:03 +02:00
5ad2126030
Small speed boost for fetching by some caching
2024-11-06 19:56:39 +02:00
6cf506cb22
PlaywrightEngine - Validate CDP URLs in all cases
2024-11-06 11:24:56 +02:00
e270b00811
Toolbelt subpackage custom functions general adjustments
2024-11-03 23:17:20 +02:00
2ff0fbb0e5
Engines utils - Navigation functions
...
- Adding doc strings and type annotations stuff.
- Making error messages more detailed.
- Renaming the `construct_websocket_url` function for more clarity.
2024-11-03 22:46:40 +02:00
c8c494e73f
Adding a function to intercept and drop unwanted requests
2024-11-03 16:25:55 +02:00
145c03daff
Big structure changes (check commit description)
...
- Moved most of the parser functions/files to the core package.
- Converted tools file to a package and made separate files for similar functions.
- Now all fetcher engines return a Response object
- Instead of selecting an engine to use and passing config to it, we have separate fetcher classes so the user can choose what to use while importing.
- I added a new custom fetcher so the user can create and use an engine.
- More...
2024-11-03 01:04:02 +02:00
Karim shoair