Crawl Errors

When redirection.io is crawling a website, many errors can occur. On this page, we will list all possible errors to help you understand what is going on and how you can solve them.

Running Out of Credits

When your account runs out of credits, the crawl will be paused. You have 24 hours to top up your account with new credits, or the crawl will be canceled.

To do so, open the project billing page and order new crawl credits.

Too Many Errors

The redirection.io crawler adapts its speed according to the website. As soon as an error occurs, the crawler will reduce its speed. An error is a response that:

• Has a status code of 401, 403, 429, or greater than 500;
• Is too slow to respond (greater than 5 seconds);
• Has an HTTP issue (invalid SSL, network error, etc.).

Moreover, the resource will be retried later, up to 3 times.

redirection.io will track the number of errors, and if the error rate exceeds 90% during the last 60 seconds, the crawl will be canceled.

To resolve this issue, you can disable the "Automatic rate limiting" setting and use a smaller value.