6699213bdc
Merge branch 'dev' into test/storage-core-coverage
2026-03-17 17:31:31 +02:00
440da11259
Merge branch 'main' into dev
2026-03-17 17:28:38 +02:00
7bfaf204b7
docs: CJK Latin spacing fixes for the JP README
2026-03-17 17:12:41 +02:00
74abd54a0b
docs: Add translations to the new sponsor
2026-03-17 17:00:32 +02:00
3d60792cbc
docs: update Zensical
2026-03-17 16:53:55 +02:00
3b543a6bdd
docs: add a new sponsor
2026-03-17 16:45:04 +02:00
012e9f9f5e
docs: images adjustments
2026-03-16 16:23:08 +02:00
f25bca75bf
docs: fixing broken links
...
Also, Fixes #199
2026-03-16 15:38:23 +02:00
ca6c1f3c1e
docs: CJK Latin spacing fixes for the Chinese README
2026-03-16 15:26:25 +02:00
72a2c8de81
Merge branch 'main' into dev
2026-03-15 22:44:16 +02:00
eb32285c49
docs: add the sponsor translated versions
2026-03-15 22:25:50 +02:00
f286575f31
docs: adding a new sponsor
2026-03-15 22:17:12 +02:00
e63e57e6dc
test(ai): add _normalize_credentials edge case coverage ( #192 )
2026-03-15 16:25:57 +02:00
656dbc7d12
Merge branch 'dev' into test/normalize-credentials-coverage
2026-03-15 16:24:59 +02:00
297423410d
test: add save/retrieve round-trip and hash/URL coverage for storage core
...
The adaptive scraping feature relies on SQLiteStorageSystem to persist
and relocate elements, but the existing tests only verified object
creation — not the actual save/retrieve workflow. This adds:
- _get_base_url: None, empty, valid URL, and case-normalization paths
- _get_hash: determinism, uniqueness, strip/lowercase, length suffix
- save/retrieve round-trip: basic, overwrite (upsert), nonexistent key
- URL-based isolation between different websites
- element_to_dict: with/without text, attributes, whitespace filtering
- _get_element_path: nested and root element paths
- Thread safety: 20 concurrent saves with result verification
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-15 17:27:10 +08:00
54fcb7c364
test(ai): add _normalize_credentials edge case coverage
2026-03-15 00:14:54 -04:00
c7f0db912c
docs: move up a new sponsor
2026-03-14 17:08:51 +02:00
184ba6d0b0
docs: Add Korean (한국어) README translation ( #187 )
2026-03-11 16:49:52 +02:00
60243a823a
docs: Add Korean (한국어) README translation
2026-03-11 14:33:58 +09:00
3836712939
docs: adjustments
2026-03-11 06:13:30 +02:00
50388bf675
docs: update sponsors page
2026-03-11 05:54:54 +02:00
d67a397ecc
docs: add the sponsor to the translated README files
2026-03-11 05:47:32 +02:00
569e317437
docs: add new sponsor
2026-03-11 05:42:18 +02:00
79a03ae7f0
docs: move up a sponsor
2026-03-11 01:11:19 +02:00
1aa150f090
docs: replace the downloads badge
2026-03-11 00:06:05 +02:00
19c21c17ee
v0.4.2 ( #183 )
2026-03-09 01:37:28 +02:00
6ec40ad19d
docs: remove a sponsor
2026-03-09 00:48:15 +02:00
f80743a97b
style: remove unused import
2026-03-09 00:35:06 +02:00
66e0e7af57
fix(agent): update the agent skill zip file with the new version
2026-03-09 00:31:57 +02:00
73832f9dfe
docs: update the website to reflect the google referer logic
2026-03-09 00:29:54 +02:00
d48ac78c79
test: update tests accordingly
2026-03-09 00:28:32 +02:00
8e3bf89e6c
fix(agent): update the agent skills docs and version
2026-03-09 00:27:10 +02:00
a929de6ca4
fix: update code and docstrings to remove the old google referer logic
2026-03-09 00:25:19 +02:00
b547e04207
ops: update playwright pinned version
2026-03-09 00:05:46 +02:00
27409927a3
build: pump up deps and browsers versions
2026-03-08 23:57:10 +02:00
aa6f96eb67
fix: return bare Google URL in referer instead of search query ( #179 )
...
Solves #172
2026-03-08 23:20:43 +02:00
b0c8448f81
Merge branch 'dev' into fix/google-referrer-spoof
2026-03-08 19:01:20 +02:00
11bc08fbaa
fix: Selector.get_all_text() doesn't get all text #167 ( #168 )
...
Fixes #167
2026-03-08 19:01:04 +02:00
68b9e128bc
fix: return bare Google URL in referer instead of search query
...
Real browsers send `https://www.google.com/ ` as the Referer header
when clicking search results, not the full search URL with query
parameters. The previous format was a fingerprinting signal that
the referer was spoofed.
Closes #172
2026-03-08 11:32:04 -04:00
7cbe566acd
Move test for get_all_text to test_parser_advanced.py
2026-03-08 08:20:30 -07:00
e045d0f73f
Merge branch 'dev' into dev
2026-03-08 16:31:25 +02:00
49db2b643b
Merge branch 'main' into dev
2026-03-08 16:27:25 +02:00
b9b5bb0c80
docs: add badges for the agent skill and Clawhub
2026-03-08 06:56:50 +02:00
68e6c3c3e8
docs: adjustments
2026-03-08 06:39:49 +02:00
d1a0b815f5
docs: update translation files with new sponsor
2026-03-08 06:27:04 +02:00
cb7e9b9ef3
docs: add new platinum sponsor
2026-03-08 06:16:48 +02:00
25de97f534
docs: update the agent skill readme
2026-03-08 03:30:04 +02:00
68da2af556
docs: update the agent skill readme
2026-03-08 03:21:23 +02:00
00ad6b4c5f
Add two ignored elements to unit test
2026-03-07 17:11:39 -08:00
5806b8e014
- Add pre-compiled XPath text selector
...
- Avoid recursion
2026-03-07 17:11:39 -08:00
Karim shoair