dac854a129
fix(parser): handle responses with empty body
...
It makes root's value None
2026-01-13 02:35:26 +02:00
ae719a9d54
fix(parser): Improve response to json conversion
2025-12-17 00:15:55 +02:00
170205599d
fix(parser): Better approach for web pages where the encoding is not always correctly declared
...
Fixes #110 and avoids defaulting to a specific encoding like #111
2025-11-22 20:24:55 +02:00
e20882af5d
style: removing dead code from the parser
2025-11-08 20:08:23 +02:00
debe03256b
refactor: Making all the codebase acceptable by PyRight
...
Also fixes #97
2025-10-05 04:03:39 +03:00
3da806210b
perf: General code restructure to not use more than needed memory
2025-09-23 18:34:54 +03:00
63a46a8e0b
fix(parser): An encoding issue with converting bytes to string on some encoding types
...
Removing that `invalid start byte` annoying bug
2025-09-23 18:32:56 +03:00
a9d05cc0ef
style: Removing dead code/docstrings and correcting type hints
2025-09-19 04:49:30 +03:00
b197f8a5f9
fix(parser): Improve selectors re function
2025-09-18 13:56:37 +03:00
570303f938
fix(parser): Make html_content and prettify return strings not bytes (depends on the encoding)
2025-09-16 01:58:51 +03:00
ce4fc31f2c
feat: Make .body return the passed content as it is without any processing
...
This makes it possible to download files and deal with non-HTML requests (ex: #81 )
2025-09-14 20:21:45 +03:00
67ca139ff9
fix: Fixes for multiple encoding issues ( #80 & #81 )
2025-09-14 20:20:29 +03:00
13d7e70cb7
refactor: Make all fetchers as an optional dependency group
...
+ Removing some dead code
2025-09-13 16:07:48 +03:00
330d03559c
style: applying the new ruff rules to all files
2025-09-13 03:22:53 +03:00
cf57af588e
docs: update doc strings with correct naming
2025-08-25 06:08:15 +03:00
2f402f4835
style: add flags for tests coverage
...
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
84337ef1ac
fix(parser): count all nested children of ignored tags in get_all_text
2025-08-16 14:24:51 +03:00
740ae815c3
perf: speeding up find_by_text and find_by_regex by 3%
2025-08-02 19:46:18 +03:00
67658f1723
perf: Speeding up below_elements and relocate by 3%
2025-08-02 19:45:43 +03:00
d1a2ecd341
perf: optimize get_all_text and adaptive logic by another 10%
2025-08-02 04:08:14 +03:00
1ec6f0e0f0
perf: optimizing find_similar method
2025-08-02 03:39:05 +03:00
aec4889d25
perf: Optimizing next and previous properties
2025-08-02 03:37:48 +03:00
548a40d850
perf: speed up get_all_text function by another 20%
2025-08-02 03:36:59 +03:00
4c4202daae
perf(parser): Speeding up css_first and xpath_first than normal ones
2025-08-01 16:41:09 +03:00
83a19f3b17
perf(parser): A lot of optimizations to speed things up
2025-08-01 06:28:52 +03:00
df48662c00
perf(parser): A lot of optimizations to speed things up
2025-08-01 05:42:32 +03:00
af5f3688c1
fix: moving types to use Union again
2025-07-30 05:39:25 +03:00
32cb76604c
style: Adjustments to the translator
2025-07-30 03:28:16 +03:00
5bb1266fa5
style: using isinstance function as the main way for type checking
2025-07-30 03:14:23 +03:00
ae9ccaec79
style: A lot of type hints correction
...
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
9415acccce
fix: shortcuts for backward compatibility
2025-07-30 01:50:48 +03:00
7300efa77e
style(parser): optimize selectors instances creation
2025-07-30 01:45:26 +03:00
e7cdd39695
style: replacing os with Pathlib and small optimizations
2025-07-30 01:15:28 +03:00
9bcb9e9d93
style: General type hints fixes and imports optimizing
2025-07-29 23:43:06 +03:00
715dfb4243
feat: add length property to Selectors to write less code
2025-07-29 06:19:45 +03:00
29b77a96c8
refactor(parser): optimize imports
2025-07-29 06:18:12 +03:00
b9c7a5af2e
refactor: replace's Selector inpt (text/body) with 1 argument called content
2025-07-29 06:08:15 +03:00
264ae02aa7
refactor: huge change, many features/class got a better naming
...
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
a5e4b91653
refactor: remove clean up function for Adaptor + make adaptor attributes accessible directly
2025-07-28 14:48:22 +03:00
297e14230b
refactor(parser): Multiple optimizations and fixes
2025-07-28 03:32:22 +03:00
d220050160
refactor(parser): Make get_all_text method 40% faster
2025-07-28 03:29:13 +03:00
b60fcbb884
refactor(Adaptor): Cleaner approach to find_similar method
...
This code is slower than before by about 2-5μs, but it's worth it.
2025-07-27 23:34:22 +03:00
ffde7e8ad9
fix(Adaptor): Add cleanup function to handle possible memory leak
2025-07-27 23:31:55 +03:00
6ae1810405
docs: improve All Adaptor class doc strings
2025-07-27 22:41:37 +03:00
93bd131bb6
fix(parser): Solve the ignored elements children issue while keeping speed
...
solves #61
2025-07-27 05:34:48 +03:00
0c8dd63f87
chore: migrating to ruff and updating pre-commit hooks
2025-04-13 17:32:00 +02:00
811cd2d9b0
refactor: make auto_match disabled by default
2025-03-26 19:25:19 +02:00
68ac9d4d05
fix(parser): Lowering memory with replacing body property with html_content
2025-03-17 05:58:40 +02:00
e5a022ec6a
fix(parser): Improving CSS selecting memory usage by ~144%
...
and speed by ~2%
2025-03-16 22:33:46 +02:00
94909099a9
fix(Parser): Change default text content to empty string instead of None
...
For compatibility with previous fix/commit
2025-03-15 00:56:19 +02:00
Karim shoair