 Karim shoair
|
297e14230b
|
refactor(parser): Multiple optimizations and fixes
|
2025-07-28 03:32:22 +03:00 |
|
|
Karim shoair
|
e35bade804
|
build: update bandit rules
|
2025-07-28 03:32:17 +03:00 |
|
|
Karim shoair
|
d220050160
|
refactor(parser): Make get_all_text method 40% faster
|
2025-07-28 03:29:13 +03:00 |
|
|
Karim shoair
|
7f11b6f59b
|
fix(storage): possible threading issue with recursion + optimizations
|
2025-07-28 00:16:58 +03:00 |
|
|
Karim shoair
|
3b0237e402
|
docs: improve All storage classes doc strings
|
2025-07-27 23:37:47 +03:00 |
|
|
Karim shoair
|
b60fcbb884
|
refactor(Adaptor): Cleaner approach to find_similar method
This code is slower than before by about 2-5μs, but it's worth it.
|
2025-07-27 23:34:22 +03:00 |
|
|
Karim shoair
|
ffde7e8ad9
|
fix(Adaptor): Add cleanup function to handle possible memory leak
|
2025-07-27 23:31:55 +03:00 |
|
|
Karim shoair
|
6ae1810405
|
docs: improve All Adaptor class doc strings
|
2025-07-27 22:41:37 +03:00 |
|
|
Karim shoair
|
93bd131bb6
|
fix(parser): Solve the ignored elements children issue while keeping speed
solves #61
|
2025-07-27 05:34:48 +03:00 |
|
|
Karim shoair
|
d1aa0be6e4
|
refactor(StealthyFetcher): Remove virtual mode and use persistent context
Solves #64 completely too
|
2025-07-27 03:41:28 +03:00 |
|
|
Karim shoair
|
172a5b4a0a
|
feat: Add an mcp server
|
2025-07-26 22:57:08 +03:00 |
|
|
Karim shoair
|
9d3b1335a9
|
build: update deps
|
2025-07-26 19:00:38 +03:00 |
|
|
Karim shoair
|
1076001e29
|
refactor: changes to be used by the mcp server
|
2025-07-14 05:38:29 +03:00 |
|
|
Karim shoair
|
f7e1b7261e
|
ops: adjust vermin to use 3.10 as the new minimum Python version
|
2025-07-14 05:38:16 +03:00 |
|
|
Karim shoair
|
ecef751468
|
fix: small fix for fetcher
|
2025-07-14 02:14:54 +03:00 |
|
|
Karim shoair
|
0e5fd502d2
|
Merge branch 'main' into dev
|
2025-07-11 02:44:10 +03:00 |
|
|
Karim shoair
|
80e3a97990
|
docs: Update README.md
|
2025-07-10 21:19:00 +03:00 |
|
|
Karim shoair
|
51b5c4dff8
|
Merge branch 'main' into dev
|
2025-07-10 21:17:39 +03:00 |
|
|
Karim shoair
|
377ad92485
|
docs: update PetroSky promo
|
2025-07-10 21:14:34 +03:00 |
|
|
Karim shoair
|
6eb9626865
|
docs: add PetroSky as sponsor
|
2025-07-10 21:12:54 +03:00 |
|
|
Karim shoair
|
8af4833c64
|
chore: add new sponsor poster
|
2025-07-10 21:00:06 +03:00 |
|
|
Karim shoair
|
96be8af1b3
|
chore: add new sponsor poster
|
2025-07-10 20:56:20 +03:00 |
|
|
Karim shoair
|
5d68de8f31
|
Merge branch 'main' into docs
|
2025-07-09 19:39:39 +03:00 |
|
|
Karim shoair
|
1c6a960f17
|
Merge branch 'main' into dev
|
2025-07-09 19:39:08 +03:00 |
|
|
Karim shoair
|
119e00e16d
|
chore: Delete the "others" issue template
|
2025-07-09 19:30:06 +03:00 |
|
|
Karim shoair
|
fe8b11155e
|
test: remove tests for Python 3.9
|
2025-07-09 16:31:05 +03:00 |
|
|
Karim shoair
|
4bba8598c2
|
build: update deps
|
2025-07-09 16:20:21 +03:00 |
|
|
Karim shoair
|
403f7bfe72
|
build: Updating PyPI classifiers
|
2025-07-07 00:10:20 +03:00 |
|
|
Karim shoair
|
939b78f3cf
|
build: adding mcp as dependency
|
2025-07-07 00:09:54 +03:00 |
|
|
Karim shoair
|
3184dd8581
|
build: set minimum version to Python 3.10
|
2025-07-07 00:09:02 +03:00 |
|
|
Karim shoair
|
2b73fdbfd7
|
fix(DynamicFetcher): Remove old argument
|
2025-07-06 05:02:16 +03:00 |
|
|
Karim shoair
|
2ecf9b0c20
|
build: Improve description
|
2025-07-06 05:01:19 +03:00 |
|
|
Karim shoair
|
63a9db21a1
|
refactor(DynamicSession): Optimization + Removing max_pages from sync version
|
2025-07-06 04:59:22 +03:00 |
|
|
Karim shoair
|
6d79927233
|
refactor(Fetcher): Fix the issue of caching impersonation state + Less code duplication
It now creates a session with each request, but at the same time, it's still faster than the fetcher in v0.2.99 by 20% with more features enabled.
|
2025-07-06 02:21:15 +03:00 |
|
|
Karim shoair
|
18e56a8c50
|
refactor: Optimizing extract command and cleaning code
|
2025-06-29 19:40:03 +03:00 |
|
|
Karim shoair
|
29e485bdf2
|
refactor: Optimizations to CLI
|
2025-06-29 19:10:52 +03:00 |
|
|
Karim shoair
|
b28f90e854
|
build: Update deps
|
2025-06-29 18:58:36 +03:00 |
|
|
Karim shoair
|
1c9e48b1c6
|
feat(extract): Adding new command to CLI options + Optimizations
Users can now fetch websites directly without code and extract full/selected HTML content as HTML, Markdown, or extract text content.
|
2025-06-29 18:56:09 +03:00 |
|
|
Karim shoair
|
ce034f9f79
|
test: remove problematic test
|
2025-06-28 18:24:08 +03:00 |
|
|
Karim shoair
|
b02f0bd236
|
ops: Adjust tox file to avoid CI issues
|
2025-06-28 18:12:38 +03:00 |
|
|
Karim shoair
|
ac3db69c47
|
ops: new approach to CI tests workflow
|
2025-06-28 17:40:19 +03:00 |
|
|
Karim shoair
|
ee1a0828a2
|
ops: Correcting cache paths
|
2025-06-27 17:16:35 +03:00 |
|
|
Karim shoair
|
d603c15212
|
ops: rewrite tests logic to cache downloaded browsers before testing
|
2025-06-27 16:02:16 +03:00 |
|
|
Karim shoair
|
4ee32af67b
|
ops: fix for GitHub's CI
|
2025-06-27 14:50:06 +03:00 |
|
|
Karim shoair
|
3578a07c0a
|
ops: fix for test deps
|
2025-06-27 14:35:12 +03:00 |
|
|
Karim shoair
|
94edb92bc3
|
ops: update hashing for the cache
|
2025-06-27 14:19:07 +03:00 |
|
|
Karim shoair
|
941ed20a15
|
ops: switch tests to use MacOS
|
2025-06-27 03:19:58 +03:00 |
|
|
Karim shoair
|
d6f8926d7a
|
ops: switch tests to Windows system instead of Linux
|
2025-06-27 03:08:42 +03:00 |
|
|
Karim shoair
|
dbf7aa7761
|
feat(DynamicFetcher): Use persistent context by default for better stealth
And might solve the sandbox issue with GitHub
|
2025-06-27 03:02:46 +03:00 |
|
|
Karim shoair
|
76248eabf6
|
ops: possible fix for sandbox issue with GitHub's CI
|
2025-06-27 03:01:27 +03:00 |
|