In today’s post, I’m going to go over the last few sections of GWT – Google Index, Crawl and Security Issues. There’s some super handy stuff for bloggers here, so let’s get started!
This section of GWT is concerned with how your site is indexed by Google – from how many pages, to how to remove URLs from being indexed.
- Index Status – Take a good ol’ look at this report to see how many pages of your site have been indexed over time by Google. You can help Google along with this by submitting a Robots.txt file, but we’ll cover that a bit more under the Crawl reports.
- Content Keywords – this is a list of keywords Google has picked up and deemed most relevant across your site. Review these along the Search Queries report to ensure Google is understanding your content properly.
- Remove URLs – Want to remove certain URLs from being indexed? This is useful for archive URLs or broken links you are unable to fix.
- Crawl Errors – This report will show you the errors Google encountered when trying to crawl your site. From my experience, it’s usually 404 errors as a result of comments being deleted (i.e. spam comments).
- Crawl Stats – This report simply shows you pages crawled per day, KB downloaded and time spent downloading a page. You might find if you have a lot of high-res images that the time spent downloading will increase. Make sure you’re compressing images, as site speed is an important factor for Google when ranking sites.
- Fetch As Google – This tool lets you input specific URLs to see how Google is reading and rendering them. Simply copy and paste the permalink (part after the main domain) and click Fetch. It will show you the Googlebot type (i.e. Desktop or mobile), the status and the date you rendered it. Note that the date shows in a US timezone. You can also opt to submit a URL to index if its not already been indexed.
- Robots.txt tester – This is a great tool for testing whether Google understands your Robots.txt. In laments terms, the robots.txt is a text file that tells Google and other search engines what to crawl on your site. You can exclude certain extensions (for example, I exclude /search/label to avoid crawl errors and duplication issues).
- Sitemaps – This is where you can submit your sitemap. This is basically a map of your site (duh), telling Google what’s there. The report will show you the number of pages submitted, and the number indexed. You can upload a new sitemap to replace the old one, but you only really need to do it regularly if your site is quite complex and has a lot of content being added very regularly.
- URL Parameters – Essentially, this report is used to tell Google how to deal with specific types of URL parameters. This is mostly relevant if you have URLs with duplicate content such as session IDs. I don’t use this report at all, but if you think it may effect your site, Google’s support page explains more.
This part of GWT tells you if your site may have been hacked or if there is any malware. Hopefully, you’ll never need to look at it, but just be aware that it’s there!