Handle 503 status from protection system

[Mystogan007]

Could you please help me with the following issue?
I'm using a crawler, and it returns a 503 Service Unavailable error when trying to access a page.

The crawler throws an exception due to the 503 status code.

I don’t know how to successfully reach the target site.
Could you advise on how to handle this?

[janbuchar]

Hello there @Mystogan007 and thanks for your interest in Crawlee! An HTTP 503 should get retried automatically, and that's pretty much all that you can do, as 503 means the service is temporarily unavailable.

Of course, there is a chance that the service detected that you're scraping it and blocked you (by returning a 503 to throw you off 🤷). In that case, you might want to read https://crawlee.dev/python/docs/guides/avoid-blocking