Analyzing the results of a crawl

Congratulations, your crawl is finished! Now, you can analyze the results.

Summary

Under the "Summary" tab, you'll find the website rating panel. This section shows the overall rating of your website.

redirection.io analyses different metrics to rate your website. Theses metrics are split in 3 categories:

• Content
• Structure
• Technical

Then, each metrics has an associated level of severity:

• Error: the issue is critical, and it must be fixed as soon as possible;
• Warning: the issue is important, and it should be fixed;
• Success: the metric analyzis has been successful.

Finally, the overall score is computed on the basis of the number of issues and their severity.

You can read more about the metrics that are computed across all the URLs of the crawl in the crawler metrics reference page.

Analysis

In the "Analysis" tab, you'll find some key metrics about the crawl.

The most important part is the alerts page. At a glance, it shows the alerts found during the crawl, and the number of URLs that are affected by these alerts.

For each alert, you can click on the details button to get more documentation and also a link to see the affected URLs.

It's fine to have some alerts, but you should try to fix them as soon as possible, or to keep the list as short as possible.

Below the alerts page, you'll find a collection of other pages that give details on key metrics, displayed in a graphical way:

• Meta tags
• Social tags
• HTML Structure
• Hreflang attributes
• Links
• Status codes
• Page size
• Timing information
• etc.

If you miss some visualization, please let us know, we are always looking for ways to improve the analysis.

Exploring the crawl URLs

In the "Browse URLs" tab, you'll find the list of all the URLs that have been discovered.

To understand the meaning of each column, you can read the crawler columns reference page.

This list is widely customizable. For example, you show or hide columns, filter or sort the way you want, in order to focus on the most important URLs for a specific analysis. You can click on each URL row in the list to open a page containing a lot more information about this URL:

In the "URL Information" panel, you can see all the data that redirection.io has collected about this resource. There is also the "Page rating" panel. It's the same as the one in the summary tab, but only for this URL.

This details page contains several other tabs:

• the "Content" tab, that shows analysis criteria about the content of the page;
• the "Structure" tab, that shows analysis criteria about the structure of the page;
• the "Technical" tab, that shows different criteria about the technical aspects of the page.

In the "Links" tabs, you'll find:

• the Inlinks panel, that shows all the crawled URLs that are linking to this one;
• the Incoming redirections panel, that shows all the URLs that are redirecting to this one;
• the Outlinks panel, that shows all the URLs that this one is linking to.

But there is another feature! Let's say you have fixed some issues for this specific page, and deployed this fixes on your production site. Obviously, you will want to check if the score has been updated to reflect the changes, but it can be a bit anoying to re-crawl the entire wesbite. Instead, you can click on the "Crawl this page now" button to trigger a crawl this URL only:

Segmentation

Another great feature of the crawler, is the ability to segment the crawl into different parts. Let's say you have a big website, with an e-commerce part, a CMS section, etc. You can "segment" the crawl to focus on only one part at a time.

The segmentation is based on the URL. You can define as many segments as you want, and for each segment you can defined a list of URL patterns.

Once the segmentation is defined, you can select what segment you want to analyze, and the dashboard will be updated to show only the data for this segment.

Segments are stored at the project level, so you can reuse them for the next crawl. Also, your team members can see the segments that you have defined. If the redirection.io project is used to crawl several websites, the segments are shared across all the crawls and may not match all the websites. In such case, you may prefer to create a separate redirection.io project for each website.

Comparison over time

In a professional environment, it is really important to monitor the evolution of your website. You can setup scheduled crawls to run the same crawl on a regular basis, be notified of each crawl results and then compare the results.

In the "Summary" tab of the crawl, several charts allow to perform a quick comparison between the current crawl and the previous ones. Usually, it will be a small variation, and it's fine. But when a big change happens (for a example, a large amount of new 404 errors is detected by the crawler), you may want to know exactly what has changed!

In the "Browse URLs" tab, you can filter the URLs by their status code, and then compare the results with a previous craw. The comparison can be made manually, by opening both crawls in two separate tabs, or automatically, by using the "Compare with another crawl" feature. This feature is available on the right of the page, under the "Filters" panel.

The comparison will be made with the same set of filters for both crawls, and redirection.io will show the differences between the two crawls: which URLs are new, which URLs have been removed, which URLs have changed their status code, etc.