c2c02dc78e
docs: updating the agent skill for Clawhub validation
2026-03-25 01:38:11 +02:00
422b4713ec
build: pump up version
2026-03-25 00:19:58 +02:00
9fee49c928
test: fixes to storage tests
2026-03-25 00:13:42 +02:00
3757cf6d77
test: add save/retrieve round-trip and core storage coverage ( #193 )
2026-03-25 00:11:42 +02:00
30ffe76cfa
Merge branch 'dev' into test/storage-core-coverage
2026-03-24 22:42:58 +02:00
3b4cae6889
Merge branch 'main' into dev
2026-03-24 22:21:07 +02:00
6635a4ad72
docs: adding a new sponsor
2026-03-24 17:13:24 +02:00
718db42ff0
Merge branch 'main' into dev
2026-03-23 16:40:12 +02:00
32cec404f6
docs: adjustment to ad plans
2026-03-23 15:00:26 +02:00
e230403232
docs: removing an old sponsor
2026-03-23 14:56:35 +02:00
df4b1ac7e3
Merge branch 'dev' into test/storage-core-coverage
2026-03-22 18:05:42 +02:00
d7c08073d4
Merge branch 'main' into dev
2026-03-22 18:05:26 +02:00
a96809a340
docs: removing an old sponsor
2026-03-22 16:48:26 +02:00
bc509e9fcd
Merge branch 'main' into dev
2026-03-19 17:01:25 +02:00
3ed59c2a84
fix(agent): update skill zip file with the latest fix
2026-03-19 17:01:05 +02:00
2e78b3f8aa
fix(skill): Fix broken markdown links in skill references ( #204 )
...
Fixes #203
2026-03-19 16:58:06 +02:00
a4f31ae205
simple md fix
2026-03-19 10:37:02 -04:00
7ca02ebb80
test: add edge case tests for filter, iterancestors, and find_similar ( #200 )
2026-03-17 22:16:15 +02:00
bf9e2da329
Merge branch 'dev' into test/edge-cases-filter-ancestors-find-similar
2026-03-17 22:15:59 +02:00
1dc0b7a1bd
fix(fetchers/content): increase the default max number of retries and raise error on max retries
...
Ref.: https://github.com/D4Vinci/Scrapling/pull/197#issuecomment-4077705587
2026-03-17 22:14:02 +02:00
17ea982162
fix(fetchers): add max retry limit to _get_page_content to prevent infinite loop ( #197 )
2026-03-17 22:09:05 +02:00
8e4e59e13e
Merge branch 'dev' into fix/page-content-infinite-loop
2026-03-17 22:08:47 +02:00
d9932f2308
fix(spider/fetcher): preserve HTTP method across retries in spider session ( #201 )
2026-03-17 22:06:01 +02:00
4c07b294ae
Merge branch 'dev' into fix/preserve-http-method-on-retry
2026-03-17 22:04:57 +02:00
27f306259e
Merge branch 'dev' into fix/page-content-infinite-loop
2026-03-17 21:55:26 +02:00
136c389787
fix(Texthandler): Replace get_all with getall to match the Selector class
2026-03-17 21:53:17 +02:00
980be18a2a
test: add coverage for TextHandler regex paths and TextHandlers.re() ( #194 )
2026-03-17 21:50:05 +02:00
5123590a15
Merge branch 'dev' into test/custom-types-coverage
2026-03-17 21:42:17 +02:00
cc6c0dbfb9
fix: adjust test to the _restore_from_checkpoint fix
2026-03-17 17:46:47 +02:00
1fb7961d42
fix: replace bare raise with return False in _restore_from_checkpoint ( #196 )
2026-03-17 17:45:52 +02:00
bcd45e47bb
Merge branch 'dev' into fix/checkpoint-restore-bare-raise
2026-03-17 17:36:11 +02:00
5eefbc6c05
Merge branch 'dev' into test/custom-types-coverage
2026-03-17 17:34:12 +02:00
6699213bdc
Merge branch 'dev' into test/storage-core-coverage
2026-03-17 17:31:31 +02:00
440da11259
Merge branch 'main' into dev
2026-03-17 17:28:38 +02:00
7bfaf204b7
docs: CJK Latin spacing fixes for the JP README
2026-03-17 17:12:41 +02:00
74abd54a0b
docs: Add translations to the new sponsor
2026-03-17 17:00:32 +02:00
3d60792cbc
docs: update Zensical
2026-03-17 16:53:55 +02:00
3b543a6bdd
docs: add a new sponsor
2026-03-17 16:45:04 +02:00
5bf921b308
fix: preserve HTTP method across retries in spider session
...
SessionManager.fetch() pops `method` from `_session_kwargs`,
which mutates the original request dict. When the engine retries
a blocked request via request.copy(), the copy no longer has
`method`, so it defaults to GET.
Steps to reproduce:
1. Yield Request(url, method="POST", data=...)
2. Target returns a response that triggers is_blocked()
3. Engine retries via request.copy() → second fetch uses GET
Fix: copy the kwargs dict before popping, so the original
request stays intact.
2026-03-17 00:53:52 +09:00
012e9f9f5e
docs: images adjustments
2026-03-16 16:23:08 +02:00
f25bca75bf
docs: fixing broken links
...
Also, Fixes #199
2026-03-16 15:38:23 +02:00
ca6c1f3c1e
docs: CJK Latin spacing fixes for the Chinese README
2026-03-16 15:26:25 +02:00
cec5acd68a
test: add edge case tests for filter, iterancestors, and find_similar
...
- test_selectors_filter.py: covers chained filter(), empty result,
all-pass predicate, and calling filter() on empty Selectors
- test_ancestor_navigation.py: covers iterancestors() order, depth,
text-node safety, root-element edge case, and find_ancestor() with
no match
- test_find_similar_advanced.py: covers similarity_threshold levels,
match_text=True behavior, ignore_attributes combinations, and
text-node safety
2026-03-16 18:01:11 +08:00
72a2c8de81
Merge branch 'main' into dev
2026-03-15 22:44:16 +02:00
eb32285c49
docs: add the sponsor translated versions
2026-03-15 22:25:50 +02:00
f286575f31
docs: adding a new sponsor
2026-03-15 22:17:12 +02:00
1e7704864e
Merge branch 'dev' into test/custom-types-coverage
2026-03-15 16:38:48 +02:00
e63e57e6dc
test(ai): add _normalize_credentials edge case coverage ( #192 )
2026-03-15 16:25:57 +02:00
656dbc7d12
Merge branch 'dev' into test/normalize-credentials-coverage
2026-03-15 16:24:59 +02:00
d3c251c1ab
fix: add max retry limit to _get_page_content to prevent infinite loop
...
Both _get_page_content and _get_async_page_content use a while-True
loop that retries page.content() on PlaywrightError with no upper
bound. If the page is in a permanently broken state (crashed tab,
closed context), this loops forever and hangs the process.
Replace with a bounded for-loop (default 10 retries = 5s), returning
an empty string if all attempts fail. This preserves the existing
retry behavior for the transient Windows issue (playwright#16108)
while preventing hangs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com >
2026-03-15 17:53:23 +08:00
Karim shoair