Crawl errors provide details about the URLs in particular website that google could not crawl successfully and returns an HTTP error code. (HTTP) Hyper text transfer protocol response status codes are returned when website visitor or search engines make a request to web server. Each and every page that you normally visit will return a status code, to give the browser an instructions and information.
These status codes are three digits numbers that returns by servers. Webmaster should have the knowledge on these errors. Let’s see what this status codes say from 100s through 500s.
100s: It gives informational status where the request is been received and process is going on.
200s: A success status. Request has received and processed.
300s: It redirects and shows the status. Request has been received but it needs to perform some additional steps to complete the request.
400s: A client error. Request is raised by the client but the page is not valid.
500s: Request was made by the client but the server fails to complete the request.
There are many different HTTP status codes but not important to SEO. The major health issues of particular website are crawl errors. For retrieving and analyzing the data google sends the spiders to crawl the web page. While crawling if the spider doesn’t find the page that a link is pointing to then it discovers as a crawl error. These broken pages and crawl errors are major indicators of poor user experience on a website. This results a great fall in rankings. Monitoring the crawl errors regularly is the first work for webmaster.
Just a small guide on how to monitor
Webmaster tool: Install google webmaster tool which is free google service that gives lots of information about the website. Working with the webmaster tool is simple. Add you site and follow the instructions.
Click diagnostics in the left sidebar menu: After login into the dashboard you can find diagnostics on the left side bar, just click on that.
Crawl errors: Select crawl errors. You can find three types of crawl errors.
HTTP errors: These will occur in small numbers and are not so common.
Not found: These are bad for site and should be deal as early as possible.
Restricted by robots.txt: Robots.txt tells search engine spiders that, which page in the site should crawl and which ones should not.
Download crawl error list: There is an option to download the crawl error information. This will make easier to manage.
Identify the IRLs: Analyze the URLs and try to check whether they are useful to you r not. Sort them and make a list.
Apply appropriate actions: According to type of error occurred take the action and solve the problem.
Review webmaster tools: Check back frequently the webmaster tools.
Remember one thing SEO is not completely about building links and having rich content but also the type of user experience that you provide to audience.