how to continue with last status, keep all the dateset, request queue

[aizimuji]

for example, last time there are 300 request pending

how to resume last status and skip the start url with all dataset , requests queue remain between many program runs

[janbuchar]

Hello @aizimuji and thank you for your interest in Crawlee! If you need to re-start the crawl where you left off, you can just run the same command, but set the CRAWLEE_PURGE_ON_START environment variable to 0, e.g., CRAWLEE_PURGE_ON_START=0 python -m my_crawler.

[vdusek]

Hey @aizimuji , you can check out the doc page Resuming paused crawl.