fix: save checkpoint before cancel_scope.cancel() on force-stop to prevent data loss

On force-stop (second Ctrl+C), cancel_scope.cancel() was called BEFORE
_save_checkpoint(). Since cancel_scope.cancel() causes all subsequent
awaits within the scope to raise Cancelled, the checkpoint write was
silently aborted:

1. _save_checkpoint() uses anyio.open_file + rename — both are await
   checkpoints that get cancelled immediately
2. self.paused never gets set to True (code after the aborted save)
3. The finally block sees 'not self.paused' and calls cleanup() which
   DELETES the previous checkpoint file

Result: a user who ran a long crawl, pressed Ctrl+C twice to force-stop,
loses their entire checkpoint irrecoverably. The old checkpoint (from
periodic saves or a previous graceful pause) is deleted, and the new
one was never written.

Fix: move the cancel_scope.cancel() call AFTER the checkpoint save.
The save completes normally, self.paused is set to True, and only then
does the scope get cancelled to abort in-flight tasks. The finally
block correctly sees paused=True and skips cleanup.

Adds 6 regression tests covering:
- Force-stop checkpoint preservation (core regression)
- Graceful pause still works
- Force-stop checkpoint is loadable
- Normal completion cleanup still works
- Force-stop without checkpoint system
- Existing checkpoint not deleted on force-stop
This commit is contained in:
voidborne-d
2026-04-03 07:05:32 +00:00
committed by d 🔹
parent a61113fc29
commit eaa0e8cae6
2 changed files with 287 additions and 5 deletions
+9 -5
View File
@@ -330,11 +330,11 @@ class CrawlerEngine:
while self._running:
if self._pause_requested:
if self._active_tasks == 0 or self._force_stop:
if self._force_stop:
log.warning(f"Force stopping with {self._active_tasks} active tasks")
tg.cancel_scope.cancel()
# Only save checkpoint if checkpoint system is enabled
# Save checkpoint BEFORE cancelling the scope.
# cancel_scope.cancel() makes all subsequent awaits
# raise Cancelled, which would abort the checkpoint
# write and leave self.paused=False — causing the
# finally block to DELETE the previous checkpoint.
if self._checkpoint_system_enabled:
await self._save_checkpoint()
self.paused = True
@@ -342,6 +342,10 @@ class CrawlerEngine:
else:
log.info("Spider stopped gracefully")
if self._force_stop:
log.warning(f"Force stopping with {self._active_tasks} active tasks")
tg.cancel_scope.cancel()
self._running = False
break