9e9ba9ab10
docs: update roadmap
2025-07-29 04:20:43 +03:00
264ae02aa7
refactor: huge change, many features/class got a better naming
...
- `Adaptor` became `Selector`
- `Adaptors` became `Selectors`
- `auto_match` argument/feature became `adaptive`
- `adaptor_arguments` argument became `selector_config`
- `automatch_domain` argument became `adaptive_domain`
- `additional_arguments` argument became `additional_args`
- `storage_adaptors` file became just `storage`
2025-07-29 04:20:23 +03:00
0c649987f8
test: remove irrelevant test case
2025-07-28 15:10:23 +03:00
a5e4b91653
refactor: remove clean up function for Adaptor + make adaptor attributes accessible directly
2025-07-28 14:48:22 +03:00
297e14230b
refactor(parser): Multiple optimizations and fixes
2025-07-28 03:32:22 +03:00
e35bade804
build: update bandit rules
2025-07-28 03:32:17 +03:00
d220050160
refactor(parser): Make get_all_text method 40% faster
2025-07-28 03:29:13 +03:00
7f11b6f59b
fix(storage): possible threading issue with recursion + optimizations
2025-07-28 00:16:58 +03:00
3b0237e402
docs: improve All storage classes doc strings
2025-07-27 23:37:47 +03:00
b60fcbb884
refactor(Adaptor): Cleaner approach to find_similar method
...
This code is slower than before by about 2-5μs, but it's worth it.
2025-07-27 23:34:22 +03:00
ffde7e8ad9
fix(Adaptor): Add cleanup function to handle possible memory leak
2025-07-27 23:31:55 +03:00
6ae1810405
docs: improve All Adaptor class doc strings
2025-07-27 22:41:37 +03:00
93bd131bb6
fix(parser): Solve the ignored elements children issue while keeping speed
...
solves #61
2025-07-27 05:34:48 +03:00
d1aa0be6e4
refactor(StealthyFetcher): Remove virtual mode and use persistent context
...
Solves #64 completely too
2025-07-27 03:41:28 +03:00
172a5b4a0a
feat: Add an mcp server
2025-07-26 22:57:08 +03:00
9d3b1335a9
build: update deps
2025-07-26 19:00:38 +03:00
1076001e29
refactor: changes to be used by the mcp server
2025-07-14 05:38:29 +03:00
f7e1b7261e
ops: adjust vermin to use 3.10 as the new minimum Python version
2025-07-14 05:38:16 +03:00
ecef751468
fix: small fix for fetcher
2025-07-14 02:14:54 +03:00
0e5fd502d2
Merge branch 'main' into dev
2025-07-11 02:44:10 +03:00
80e3a97990
docs: Update README.md
2025-07-10 21:19:00 +03:00
51b5c4dff8
Merge branch 'main' into dev
2025-07-10 21:17:39 +03:00
377ad92485
docs: update PetroSky promo
2025-07-10 21:14:34 +03:00
6eb9626865
docs: add PetroSky as sponsor
2025-07-10 21:12:54 +03:00
8af4833c64
chore: add new sponsor poster
2025-07-10 21:00:06 +03:00
96be8af1b3
chore: add new sponsor poster
2025-07-10 20:56:20 +03:00
1c6a960f17
Merge branch 'main' into dev
2025-07-09 19:39:08 +03:00
119e00e16d
chore: Delete the "others" issue template
2025-07-09 19:30:06 +03:00
fe8b11155e
test: remove tests for Python 3.9
2025-07-09 16:31:05 +03:00
4bba8598c2
build: update deps
2025-07-09 16:20:21 +03:00
403f7bfe72
build: Updating PyPI classifiers
2025-07-07 00:10:20 +03:00
939b78f3cf
build: adding mcp as dependency
2025-07-07 00:09:54 +03:00
3184dd8581
build: set minimum version to Python 3.10
2025-07-07 00:09:02 +03:00
2b73fdbfd7
fix(DynamicFetcher): Remove old argument
2025-07-06 05:02:16 +03:00
2ecf9b0c20
build: Improve description
2025-07-06 05:01:19 +03:00
63a9db21a1
refactor(DynamicSession): Optimization + Removing max_pages from sync version
2025-07-06 04:59:22 +03:00
6d79927233
refactor(Fetcher): Fix the issue of caching impersonation state + Less code duplication
...
It now creates a session with each request, but at the same time, it's still faster than the fetcher in v0.2.99 by 20% with more features enabled.
2025-07-06 02:21:15 +03:00
18e56a8c50
refactor: Optimizing extract command and cleaning code
2025-06-29 19:40:03 +03:00
29e485bdf2
refactor: Optimizations to CLI
2025-06-29 19:10:52 +03:00
b28f90e854
build: Update deps
2025-06-29 18:58:36 +03:00
1c9e48b1c6
feat(extract): Adding new command to CLI options + Optimizations
...
Users can now fetch websites directly without code and extract full/selected HTML content as HTML, Markdown, or extract text content.
2025-06-29 18:56:09 +03:00
ce034f9f79
test: remove problematic test
2025-06-28 18:24:08 +03:00
b02f0bd236
ops: Adjust tox file to avoid CI issues
2025-06-28 18:12:38 +03:00
ac3db69c47
ops: new approach to CI tests workflow
2025-06-28 17:40:19 +03:00
ee1a0828a2
ops: Correcting cache paths
2025-06-27 17:16:35 +03:00
d603c15212
ops: rewrite tests logic to cache downloaded browsers before testing
2025-06-27 16:02:16 +03:00
4ee32af67b
ops: fix for GitHub's CI
2025-06-27 14:50:06 +03:00
3578a07c0a
ops: fix for test deps
2025-06-27 14:35:12 +03:00
94edb92bc3
ops: update hashing for the cache
2025-06-27 14:19:07 +03:00
941ed20a15
ops: switch tests to use MacOS
2025-06-27 03:19:58 +03:00
Karim shoair