fix(spiders): use os.replace for atomic checkpoint/cache writes on Windows

CheckpointManager.save() and ResponseCacheManager.put() write to a temp
file and then move it into place with Path.rename(). On Windows, os.rename
cannot overwrite an existing destination and raises FileExistsError
(WinError 183), so every write after the first one fails: checkpoint
saving raises and breaks resume, while the development response cache
swallows the error and keeps returning the stale entry.

Path.replace() (os.replace) overwrites the destination atomically on every
platform and behaves identically to rename() on POSIX, so this is a no-op
on Linux and macOS and only fixes the broken overwrite on Windows.

Add a regression test for the cache overwrite path; the checkpoint
overwrite is already covered by test_multiple_saves_overwrite.
This commit is contained in:
Ahmed Elshahat
2026-06-07 04:24:12 +03:00
parent 53ef32c723
commit f0db3d7d14
3 changed files with 23 additions and 2 deletions
+1 -1
View File
@@ -64,7 +64,7 @@ class ResponseCacheManager:
async with await anyio.open_file(temp_path, "wb") as f:
await f.write(serialized)
await temp_path.rename(self._cache_path(fingerprint))
await temp_path.replace(self._cache_path(fingerprint))
except Exception as e:
if await temp_path.exists():
await temp_path.unlink()