cfc667f7dc
refactor(fetchers)!: Replace Camoufox with patchright and many optimizations
...
- DynamicFetcher became 20% faster
- StealthyFetcher became 99% faster
- Scrapling size decreased
- Code became ~400 lines shorter
- Most importantly, scrapling is more stable and reliable now.
- Less confusing for new users.
- More...
2025-12-26 03:21:13 +02:00
ac2d4d3833
fix: use the correct signature for the stealthy_fetch shortcut
2025-12-02 02:15:42 +02:00
9b3309f825
fix(shell): use correct argument name in signatures
2025-11-26 02:22:29 +02:00
8940bdeb55
fix(shell): dynamically build the signature of shortcuts after last changes
2025-11-25 21:23:06 +02:00
5376c97996
fix(fetcher): Remove non-keywords arguments
2025-11-25 20:25:09 +02:00
f0b01fc253
refactor(browser fetchers): Make all the type hints dynamic + Faster validation
...
Made the code shorter by an additional ~200 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-25 20:18:36 +02:00
357b170e1c
refactor(requests): Make all the type hints dynamic
...
Made the code shorter by about ~500 lines and easier to maintain in return for making the arguments autocompletion bad for shells that don't check for dynamic type hints like IPython.
2025-11-23 20:48:51 +02:00
fb80beba04
feat(TextHandler): Add argument to clean method to remove html entities
2025-11-17 01:36:06 +02:00
3565f9d500
feat(fetcher): Make impersonate able to randomize fingerprint
2025-11-16 16:33:58 +02:00
debe03256b
refactor: Making all the codebase acceptable by PyRight
...
Also fixes #97
2025-10-05 04:03:39 +03:00
eedfa855ab
fix: Fixes for the type checking in the main init file
...
and a bit of cleaning
2025-10-01 03:50:39 +03:00
f6c122b87f
style: Removing dead code/docstrings
2025-09-29 05:09:23 +03:00
8ad7cd6343
docs: Update all pages/docstrings to reflect recent changes
2025-09-29 03:57:40 +03:00
74f20d2a0e
refactor: better implementation for the mcp mode
2025-09-28 20:14:04 +03:00
4a661b4875
feat: Make mcp able to use http transport
2025-09-28 04:54:35 +03:00
3da806210b
perf: General code restructure to not use more than needed memory
2025-09-23 18:34:54 +03:00
2d704b2a8b
fix(shell): Fixing a bug with content converting
2025-09-19 04:50:15 +03:00
a9d05cc0ef
style: Removing dead code/docstrings and correcting type hints
2025-09-19 04:49:30 +03:00
bf5fe6d451
fix(shell): Fix view command edge case
2025-09-19 04:21:15 +03:00
504ecdf7dc
fix(shell): Fix parsing curl --data-raw
2025-09-19 04:20:16 +03:00
05c2d9aa00
feat(CLI): Make the session classes available in the shell
2025-09-15 23:34:36 +03:00
1547366c85
fix: fixing body issue with cli view command
2025-09-15 21:32:57 +03:00
f848e6b7f2
fix: Fixes to handle the new extra deps group
2025-09-15 00:19:19 +03:00
67ca139ff9
fix: Fixes for multiple encoding issues ( #80 & #81 )
2025-09-14 20:20:29 +03:00
0b3509a295
fix(shell): Fix a small typo
2025-09-14 05:05:55 +03:00
13d7e70cb7
refactor: Make all fetchers as an optional dependency group
...
+ Removing some dead code
2025-09-13 16:07:48 +03:00
330d03559c
style: applying the new ruff rules to all files
2025-09-13 03:22:53 +03:00
af9a62a539
fix: fix for shell imported by mistake issue ( #76 )
2025-09-02 15:23:15 +03:00
2f402f4835
style: add flags for tests coverage
...
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
13700e2692
fix: improve error handling
2025-08-15 04:52:11 +03:00
83a19f3b17
perf(parser): A lot of optimizations to speed things up
2025-08-01 06:28:52 +03:00
f93348b915
style: remove unused code
2025-08-01 05:43:31 +03:00
df48662c00
perf(parser): A lot of optimizations to speed things up
2025-08-01 05:42:32 +03:00
af5f3688c1
fix: moving types to use Union again
2025-07-30 05:39:25 +03:00
27658b33ff
fix: fix invalid return type
2025-07-30 03:39:33 +03:00
32cb76604c
style: Adjustments to the translator
2025-07-30 03:28:16 +03:00
5bb1266fa5
style: using isinstance function as the main way for type checking
2025-07-30 03:14:23 +03:00
ae9ccaec79
style: A lot of type hints correction
...
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
ca12a11b7e
style(utils): optimize imports
2025-07-30 02:08:01 +03:00
db781dcc32
style: Removing dead code
2025-07-30 02:03:39 +03:00
3c5de8e0f2
perf: Speed up clean functions
2025-07-30 02:03:27 +03:00
e7cdd39695
style: replacing os with Pathlib and small optimizations
2025-07-30 01:15:28 +03:00
18660f8132
style: type hints corrections and docstrings
2025-07-30 00:32:39 +03:00
ba585c0adc
perf: imports optimizing
2025-07-30 00:22:06 +03:00
9bcb9e9d93
style: General type hints fixes and imports optimizing
2025-07-29 23:43:06 +03:00
9ce3279488
fix(TextHandler): Increase speed of clean method 5 times
2025-07-29 22:31:03 +03:00
264ae02aa7
refactor: huge change, many features/class got a better naming
...
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
7f11b6f59b
fix(storage): possible threading issue with recursion + optimizations
2025-07-28 00:16:58 +03:00
3b0237e402
docs: improve All storage classes doc strings
2025-07-27 23:37:47 +03:00
d1aa0be6e4
refactor(StealthyFetcher): Remove virtual mode and use persistent context
...
Solves #64 completely too
2025-07-27 03:41:28 +03:00
Karim shoair