Repository navigation
Conversation
KeyValueStore.purge()
| @@ -141,12 +141,11 @@ async def drop(self) -> None: | |||
| storage_instance_manager = service_locator.storage_instance_manager | |||
| storage_instance_manager.remove_from_cache(self) | |||
There was a problem hiding this comment.
Suggestion: The store leaves the instance cache before _clear_store_and_cache(). When the backend drop fails, the autosaved values are restored but the store is no longer cached. A later KeyValueStore.open() then creates a second instance that shares _autosaved_values[id] but has its own _autosave_lock. Crawlee JS removes the store from the cache only after backend.drop() succeeds. Moving remove_from_cache(self) after the clear would do the same here.
✍️ Drafted by Claude Code
| # The read gets time to load the record while the purge is parked. | ||
| read_task = asyncio.create_task(kvs.get_auto_saved_value('state', {'count': 0})) | ||
| await asyncio.wait([read_task], timeout=0.1) | ||
| release_purge.set() |
There was a problem hiding this comment.
Nit: The 0.1 s wait gives the read time to run, but nothing checks that it was blocked. An assert makes the check explicit. A timeout on purge_started.wait() (line 1457) would also make a regression fail fast instead of hanging.
| release_purge.set() | |
| assert not read_task.done() | |
| release_purge.set() |
✍️ Drafted by Claude Code
|
Note: On its own, this PR still lets a ✍️ Drafted by Claude Code |
Description
KeyValueStore.purge()now also drops the values thatget_auto_saved_value()anduse_statekeep in memory for the store, the same asdrop(). A value obtained before the purge isn't saved anymore, and the nextget_auto_saved_value()oruse_statecall starts from the default value. A later save used to write these values back into the purged store.If a purge or a drop fails or is cancelled, the values stay and keep being saved. A
get_auto_saved_value()call made during a purge or a drop waits for it to finish.Depends on #2466. Without it, a
PERSIST_STATEsave that started before the purge can still write the value back.Testing
get_auto_saved_value(), and a failed or cancelled purge and drop.✍️ Drafted by Claude Code