fix: save checkpoint before cancel_scope.cancel() on force-stop to prevent data loss
On force-stop (second Ctrl+C), cancel_scope.cancel() was called BEFORE _save_checkpoint(). Since cancel_scope.cancel() causes all subsequent awaits within the scope to raise Cancelled, the checkpoint write was silently aborted: 1. _save_checkpoint() uses anyio.open_file + rename — both are await checkpoints that get cancelled immediately 2. self.paused never gets set to True (code after the aborted save) 3. The finally block sees 'not self.paused' and calls cleanup() which DELETES the previous checkpoint file Result: a user who ran a long crawl, pressed Ctrl+C twice to force-stop, loses their entire checkpoint irrecoverably. The old checkpoint (from periodic saves or a previous graceful pause) is deleted, and the new one was never written. Fix: move the cancel_scope.cancel() call AFTER the checkpoint save. The save completes normally, self.paused is set to True, and only then does the scope get cancelled to abort in-flight tasks. The finally block correctly sees paused=True and skips cleanup. Adds 6 regression tests covering: - Force-stop checkpoint preservation (core regression) - Graceful pause still works - Force-stop checkpoint is loadable - Normal completion cleanup still works - Force-stop without checkpoint system - Existing checkpoint not deleted on force-stop
This commit is contained in:
@@ -330,11 +330,11 @@ class CrawlerEngine:
|
||||
while self._running:
|
||||
if self._pause_requested:
|
||||
if self._active_tasks == 0 or self._force_stop:
|
||||
if self._force_stop:
|
||||
log.warning(f"Force stopping with {self._active_tasks} active tasks")
|
||||
tg.cancel_scope.cancel()
|
||||
|
||||
# Only save checkpoint if checkpoint system is enabled
|
||||
# Save checkpoint BEFORE cancelling the scope.
|
||||
# cancel_scope.cancel() makes all subsequent awaits
|
||||
# raise Cancelled, which would abort the checkpoint
|
||||
# write and leave self.paused=False — causing the
|
||||
# finally block to DELETE the previous checkpoint.
|
||||
if self._checkpoint_system_enabled:
|
||||
await self._save_checkpoint()
|
||||
self.paused = True
|
||||
@@ -342,6 +342,10 @@ class CrawlerEngine:
|
||||
else:
|
||||
log.info("Spider stopped gracefully")
|
||||
|
||||
if self._force_stop:
|
||||
log.warning(f"Force stopping with {self._active_tasks} active tasks")
|
||||
tg.cancel_scope.cancel()
|
||||
|
||||
self._running = False
|
||||
break
|
||||
|
||||
|
||||
Reference in New Issue
Block a user