13d7e70cb7
refactor: Make all fetchers as an optional dependency group
...
+ Removing some dead code
2025-09-13 16:07:48 +03:00
330d03559c
style: applying the new ruff rules to all files
2025-09-13 03:22:53 +03:00
cf57af588e
docs: update doc strings with correct naming
2025-08-25 06:08:15 +03:00
2f402f4835
style: add flags for tests coverage
...
- Some are already tested but the coverage report can't see it.
- Some are not necessary to test or too hard to test on GitHub's CI
2025-08-17 01:02:14 +03:00
84337ef1ac
fix(parser): count all nested children of ignored tags in get_all_text
2025-08-16 14:24:51 +03:00
740ae815c3
perf: speeding up find_by_text and find_by_regex by 3%
2025-08-02 19:46:18 +03:00
67658f1723
perf: Speeding up below_elements and relocate by 3%
2025-08-02 19:45:43 +03:00
d1a2ecd341
perf: optimize get_all_text and adaptive logic by another 10%
2025-08-02 04:08:14 +03:00
1ec6f0e0f0
perf: optimizing find_similar method
2025-08-02 03:39:05 +03:00
aec4889d25
perf: Optimizing next and previous properties
2025-08-02 03:37:48 +03:00
548a40d850
perf: speed up get_all_text function by another 20%
2025-08-02 03:36:59 +03:00
4c4202daae
perf(parser): Speeding up css_first and xpath_first than normal ones
2025-08-01 16:41:09 +03:00
83a19f3b17
perf(parser): A lot of optimizations to speed things up
2025-08-01 06:28:52 +03:00
df48662c00
perf(parser): A lot of optimizations to speed things up
2025-08-01 05:42:32 +03:00
af5f3688c1
fix: moving types to use Union again
2025-07-30 05:39:25 +03:00
32cb76604c
style: Adjustments to the translator
2025-07-30 03:28:16 +03:00
5bb1266fa5
style: using isinstance function as the main way for type checking
2025-07-30 03:14:23 +03:00
ae9ccaec79
style: A lot of type hints correction
...
Since we are using Py3.10 as minimum version now, we remove Union when possible
2025-07-30 02:46:57 +03:00
9415acccce
fix: shortcuts for backward compatibility
2025-07-30 01:50:48 +03:00
7300efa77e
style(parser): optimize selectors instances creation
2025-07-30 01:45:26 +03:00
e7cdd39695
style: replacing os with Pathlib and small optimizations
2025-07-30 01:15:28 +03:00
9bcb9e9d93
style: General type hints fixes and imports optimizing
2025-07-29 23:43:06 +03:00
715dfb4243
feat: add length property to Selectors to write less code
2025-07-29 06:19:45 +03:00
29b77a96c8
refactor(parser): optimize imports
2025-07-29 06:18:12 +03:00
b9c7a5af2e
refactor: replace's Selector inpt (text/body) with 1 argument called content
2025-07-29 06:08:15 +03:00
264ae02aa7
refactor: huge change, many features/class got a better naming
...
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
a5e4b91653
refactor: remove clean up function for Adaptor + make adaptor attributes accessible directly
2025-07-28 14:48:22 +03:00
297e14230b
refactor(parser): Multiple optimizations and fixes
2025-07-28 03:32:22 +03:00
d220050160
refactor(parser): Make get_all_text method 40% faster
2025-07-28 03:29:13 +03:00
b60fcbb884
refactor(Adaptor): Cleaner approach to find_similar method
...
This code is slower than before by about 2-5μs, but it's worth it.
2025-07-27 23:34:22 +03:00
ffde7e8ad9
fix(Adaptor): Add cleanup function to handle possible memory leak
2025-07-27 23:31:55 +03:00
6ae1810405
docs: improve All Adaptor class doc strings
2025-07-27 22:41:37 +03:00
93bd131bb6
fix(parser): Solve the ignored elements children issue while keeping speed
...
solves #61
2025-07-27 05:34:48 +03:00
0c8dd63f87
chore: migrating to ruff and updating pre-commit hooks
2025-04-13 17:32:00 +02:00
811cd2d9b0
refactor: make auto_match disabled by default
2025-03-26 19:25:19 +02:00
68ac9d4d05
fix(parser): Lowering memory with replacing body property with html_content
2025-03-17 05:58:40 +02:00
e5a022ec6a
fix(parser): Improving CSS selecting memory usage by ~144%
...
and speed by ~2%
2025-03-16 22:33:46 +02:00
94909099a9
fix(Parser): Change default text content to empty string instead of None
...
For compatibility with previous fix/commit
2025-03-15 00:56:19 +02:00
3d2418b67a
feat(Fetcher): Add the redirections history to Fetcher
...
Feature requested in #32
2025-02-22 16:58:54 +02:00
390bd51cb1
fix(TextHandler): correcting case_sensitive argument logic
...
This is awkward, I can't believe how this slipped from me!
2025-02-22 15:20:11 +02:00
7c35341d85
refactor(parser/get_all_text): Cleaner and a bit faster implementation
2025-01-31 02:47:49 +02:00
a6fd25f64c
fix(parser): fix traverse selectors
2025-01-31 01:57:19 +02:00
a8d475afb0
feat(parser): Make all returned html as TextHandler
...
So you can do regex on raw html if wanted etc...
2025-01-31 01:38:48 +02:00
3330e9f475
refactor(parser/automatch): Cleaner and faster implementation
...
~5% speed boost
2025-01-31 01:34:02 +02:00
37a31a120e
refactor(parser/find_by_regex): Better implementation for ~60% speed boost
2025-01-31 01:19:11 +02:00
2c6142d96d
refactor(parser/find_by_text): Better implementation for ~20% speed boost
2025-01-31 01:12:00 +02:00
1962c11f64
fix(parser/find_all): Logic issues made conditions used sometimes in a (and) fashion and other times (OR)
...
Also, the speed boost is 6-14% not 95% as I said first :(
2025-01-31 00:23:54 +02:00
57090f90a4
refactor(parser/find_all): about ~95% speed increase
...
Boom!
2025-01-30 23:39:46 +02:00
f37538320f
feat(parser): adding below_elements method
2025-01-30 23:05:10 +02:00
e507778c48
feat(Parser): Set regex methods new return types as TextHandlers
2025-01-30 02:27:38 +02:00
Karim shoair