Welcome back! (If you haven’t read part 1 of my guide to Webmaster Tools, scoot on over *here* before reading on!)

In today’s post, I’m going to go over the last few sections of GWT – Google Index, Crawl and Security Issues. There’s some super handy stuff for bloggers here, so let’s get started!

Google Index

This section of GWT is concerned with how your site is indexed by Google – from how many pages, to how to remove URLs from being indexed.

  • Index Status – Take a good ol’ look at this report to see how many pages of your site have been indexed over time by Google. You can help Google along with this by submitting a Robots.txt file, but we’ll cover that a bit more under the Crawl reports.
You can see below that just under 400 pages of my site have been crawled. This is awesome as this is pretty much all of the pages, give or take a few newer posts that will take a little longer to be added.
 
  • Content Keywords – this is a list of keywords Google has picked up and deemed most relevant across your site. Review these along the Search Queries report to ensure Google is understanding your content properly.
  • Blocked Resources – This is a fairly recent addition to GWT, of which I have to admit I’ve never used! However, here’s some info on Search Engine Land that tells you a bit more about its use. TLDR; It’s basically a report to tell you what images, CSS, javascript etc are being blocked from Google.
  • Remove URLs – Want to remove certain URLs from being indexed? This is useful for archive URLs or broken links you are unable to fix.

Crawl

The reports you’ll find under the Crawl section include any errors that Google came across crawling your site, the stats for number of pages crawled, and more.
  • Crawl Errors – This report will show you the errors Google encountered when trying to crawl your site. From my experience, it’s usually 404 errors as a result of comments being deleted (i.e. spam comments).
You can add these to the Remove URLs list if you know longer want Google to try and crawl them.
  • Crawl Stats – This report simply shows you pages crawled per day, KB downloaded and time spent downloading a page. You might find if you have a lot of high-res images that the time spent downloading will increase. Make sure you’re compressing images, as site speed is an important factor for Google when ranking sites.
  • Fetch As Google – This tool lets you input specific URLs to see how Google is reading and rendering them. Simply copy and paste the permalink (part after the main domain) and click Fetch. It will show you the Googlebot type (i.e. Desktop or mobile), the status and the date you rendered it. Note that the date shows in a US timezone. You can also opt to submit a URL to index if its not already been indexed.
  • Robots.txt tester – This is a great tool for testing whether Google understands your Robots.txt. In laments terms, the robots.txt is a text file that tells Google and other search engines what to crawl on your site. You can exclude certain extensions (for example, I exclude /search/label to avoid crawl errors and duplication issues).
  • Sitemaps – This is where you can submit your sitemap. This is basically a map of your site (duh), telling Google what’s there. The report will show you the number of pages submitted, and the number indexed. You can upload a new sitemap to replace the old one, but you only really need to do it regularly if your site is quite complex and has a lot of content being added very regularly.
  • URL Parameters – Essentially, this report is used to tell Google how to deal with specific types of URL parameters. This is mostly relevant if you have URLs with duplicate content such as session IDs. I don’t use this report at all, but if you think it may effect your site, Google’s support page explains more.

Security Issues

This part of GWT tells you if your site may have been hacked or if there is any malware. Hopefully, you’ll never need to look at it, but just be aware that it’s there!

So that’s it – an overview of GWT for bloggers. You may find you never use it, but it’s useful if you want to run a bit of an audit of your site. 

Read Part 1 here