Google Crawl Errors

If you are using Google Analytics to monitor the performance of a website, you can determine whether Google encountered errors crawling the site using Google's Webmaster Tools. From the Webmaster Tools page, select the site, then click on Crawl Errors under Crawl. Or you can use a URL similar to https://www.google.com/webmasters/tools/crawl-errors?hl=en&siteUrl= followed by the URL for the home page of the site you wish to check. E.g.:

https://www.google.com/webmasters/tools/crawl-errors?hl=en&siteUrl=http://support.moonpoint.com/

For the Crawl Errors report, you will see the number of Soft 404 errors, the number of URLs for which access was denied, e.g. for password protected pages, and the number of "not found" errors, e.g., if you have a URL pointing to a page on your site that doesn't exist, perhaps because the page was removed or the URL was mistyped.

Crawl Errors

You can click on Soft 404, Access Denied, or Not found to see a graph of the number of such errors for the past 90 days. You will see a Download button beneath a graph that will allow you to download the data for analysis with a spreadsheet program, etc. The data can be downloaded in comma-separated values (CSV), which most spreadsheet programs can handle, or Google Docs format.

Crawl Errors - Not found

You can use the downloaded data to locate the files on your site that contain the URLs producing the "Not found" results when the Googlebot crawls your site.

E.g., when I examined the crawl errors for my site, the first 3 shown were as follows:

URLResponse CodeNews ErrorDetected CategoryPlatformLast Crawled
http://support.moonpoint.com/network/email/clients/pine/pine-change-from-address.html 404 3/15/14Not foundDesktop 5/5/15
http://support.moonpoint.com/www.laplink.com 404 4/11/14Not foundDesktop 5/5/15
http://support.moonpoint.com/downloads/smf/mods/Nao 404 4/2/14Not foundDesktop 4/26/15

For the first one I realized I should have used .php instead of .html for the URL, since the Changing Pine "From" Address article I created was a PHP page. For the next entry, it appeared I had mistyped a URL for laplink.com, so I started at the root directory for the site and used grep -rl '/www.laplink.com' --include="*.php" to search through the files on the site for any referencing www.laplink.com. I found I had made a typo within the <a href= HTML code pointing to the site. For the third entry, I had omitted the actual link after the <a href=", putting "Nao" there by mistake. So I was able to correct pages where Google was getting a "404 Not found" error when attempting to follow links I had in the pages.

 

TechRabbit ad 300x250 newegg.com

Justdeals Daily Electronics Deals1x1 px

Valid HTML 4.01 Transitional

Created: Sunday May 10, 2015