Commit Graph

2 Commits

Author SHA1 Message Date
karesansui 5bf921b308 fix: preserve HTTP method across retries in spider session
SessionManager.fetch() pops `method` from `_session_kwargs`,
which mutates the original request dict. When the engine retries
a blocked request via request.copy(), the copy no longer has
`method`, so it defaults to GET.

Steps to reproduce:
1. Yield Request(url, method="POST", data=...)
2. Target returns a response that triggers is_blocked()
3. Engine retries via request.copy() → second fetch uses GET

Fix: copy the kwargs dict before popping, so the original
request stays intact.
2026-03-17 00:53:52 +09:00
Karim shoair d247b5b691 test: add tests for the spiders system
It's generated by Opus on Claude Code. It's very good as per my review and instructions, but I will have another look later.
2026-01-19 03:46:00 +02:00