27658b33ff
fix: fix invalid return type
2025-07-30 03:39:33 +03:00
32cb76604c
style: Adjustments to the translator
2025-07-30 03:28:16 +03:00
5bb1266fa5
style: using isinstance function as the main way for type checking
2025-07-30 03:14:23 +03:00
ae9ccaec79
style: A lot of type hints correction
...
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
ca12a11b7e
style(utils): optimize imports
2025-07-30 02:08:01 +03:00
db781dcc32
style: Removing dead code
2025-07-30 02:03:39 +03:00
3c5de8e0f2
perf: Speed up clean functions
2025-07-30 02:03:27 +03:00
9415acccce
fix: shortcuts for backward compatibility
2025-07-30 01:50:48 +03:00
7300efa77e
style(parser): optimize selectors instances creation
2025-07-30 01:45:26 +03:00
e7cdd39695
style: replacing os with Pathlib and small optimizations
2025-07-30 01:15:28 +03:00
18660f8132
style: type hints corrections and docstrings
2025-07-30 00:32:39 +03:00
ba585c0adc
perf: imports optimizing
2025-07-30 00:22:06 +03:00
8ca940d768
style: removing dead code
2025-07-29 23:43:31 +03:00
9bcb9e9d93
style: General type hints fixes and imports optimizing
2025-07-29 23:43:06 +03:00
3d07a1533e
refactor: cleaner code for the fetcher's lazy loader
2025-07-29 22:43:12 +03:00
9ce3279488
fix(TextHandler): Increase speed of clean method 5 times
2025-07-29 22:31:03 +03:00
715dfb4243
feat: add length property to Selectors to write less code
2025-07-29 06:19:45 +03:00
29b77a96c8
refactor(parser): optimize imports
2025-07-29 06:18:12 +03:00
b9c7a5af2e
refactor: replace's Selector inpt (text/body) with 1 argument called content
2025-07-29 06:08:15 +03:00
264ae02aa7
refactor: huge change, many features/class got a better naming
...
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
a5e4b91653
refactor: remove clean up function for Adaptor + make adaptor attributes accessible directly
2025-07-28 14:48:22 +03:00
297e14230b
refactor(parser): Multiple optimizations and fixes
2025-07-28 03:32:22 +03:00
d220050160
refactor(parser): Make get_all_text method 40% faster
2025-07-28 03:29:13 +03:00
7f11b6f59b
fix(storage): possible threading issue with recursion + optimizations
2025-07-28 00:16:58 +03:00
3b0237e402
docs: improve All storage classes doc strings
2025-07-27 23:37:47 +03:00
b60fcbb884
refactor(Adaptor): Cleaner approach to find_similar method
...
This code is slower than before by about 2-5μs, but it's worth it.
2025-07-27 23:34:22 +03:00
ffde7e8ad9
fix(Adaptor): Add cleanup function to handle possible memory leak
2025-07-27 23:31:55 +03:00
6ae1810405
docs: improve All Adaptor class doc strings
2025-07-27 22:41:37 +03:00
93bd131bb6
fix(parser): Solve the ignored elements children issue while keeping speed
...
solves #61
2025-07-27 05:34:48 +03:00
d1aa0be6e4
refactor(StealthyFetcher): Remove virtual mode and use persistent context
...
Solves #64 completely too
2025-07-27 03:41:28 +03:00
172a5b4a0a
feat: Add an mcp server
2025-07-26 22:57:08 +03:00
1076001e29
refactor: changes to be used by the mcp server
2025-07-14 05:38:29 +03:00
ecef751468
fix: small fix for fetcher
2025-07-14 02:14:54 +03:00
3184dd8581
build: set minimum version to Python 3.10
2025-07-07 00:09:02 +03:00
2b73fdbfd7
fix(DynamicFetcher): Remove old argument
2025-07-06 05:02:16 +03:00
63a9db21a1
refactor(DynamicSession): Optimization + Removing max_pages from sync version
2025-07-06 04:59:22 +03:00
6d79927233
refactor(Fetcher): Fix the issue of caching impersonation state + Less code duplication
...
It now creates a session with each request, but at the same time, it's still faster than the fetcher in v0.2.99 by 20% with more features enabled.
2025-07-06 02:21:15 +03:00
18e56a8c50
refactor: Optimizing extract command and cleaning code
2025-06-29 19:40:03 +03:00
29e485bdf2
refactor: Optimizations to CLI
2025-06-29 19:10:52 +03:00
1c9e48b1c6
feat(extract): Adding new command to CLI options + Optimizations
...
Users can now fetch websites directly without code and extract full/selected HTML content as HTML, Markdown, or extract text content.
2025-06-29 18:56:09 +03:00
dbf7aa7761
feat(DynamicFetcher): Use persistent context by default for better stealth
...
And might solve the sandbox issue with GitHub
2025-06-27 03:02:46 +03:00
8395db9121
fix(controllers): Fix issue with async adding stealth scripts
2025-06-27 01:39:32 +03:00
37e842b796
fix(controllers): Fix issue with adding stealth scripts
2025-06-27 01:04:58 +03:00
53318f3a2a
fix(StealthyFetcher): Fix passed proxy type
2025-06-26 16:36:30 +03:00
8d44e9c51c
fix(proxy): fix proxy unpacking
2025-06-26 15:49:53 +03:00
38851698ad
refactor: optimize imports and docstrings correction
2025-06-25 23:15:32 +03:00
a41be48047
refactor: optimize imports and docstrings correction
2025-06-25 21:36:53 +03:00
a3a5e34408
refactor: optimize imports and docstrings correction
2025-06-25 21:36:43 +03:00
724e3a648e
fix(FetcherSession): Use Sentinel pattern to solve arguments precedence issue
2025-06-25 03:53:54 +03:00
63ff12bcd6
docs: correcting a small mistake
2025-06-23 03:38:34 +03:00
Karim shoair