fix(spiders): use os.replace for atomic checkpoint/cache writes on Windows
CheckpointManager.save() and ResponseCacheManager.put() write to a temp file and then move it into place with Path.rename(). On Windows, os.rename cannot overwrite an existing destination and raises FileExistsError (WinError 183), so every write after the first one fails: checkpoint saving raises and breaks resume, while the development response cache swallows the error and keeps returning the stale entry. Path.replace() (os.replace) overwrites the destination atomically on every platform and behaves identically to rename() on POSIX, so this is a no-op on Linux and macOS and only fixes the broken overwrite on Windows. Add a regression test for the cache overwrite path; the checkpoint overwrite is already covered by test_multiple_saves_overwrite.
This commit is contained in:
@@ -64,7 +64,7 @@ class ResponseCacheManager:
|
||||
async with await anyio.open_file(temp_path, "wb") as f:
|
||||
await f.write(serialized)
|
||||
|
||||
await temp_path.rename(self._cache_path(fingerprint))
|
||||
await temp_path.replace(self._cache_path(fingerprint))
|
||||
except Exception as e:
|
||||
if await temp_path.exists():
|
||||
await temp_path.unlink()
|
||||
|
||||
Reference in New Issue
Block a user