Indexing section of the Google Search Console: How do I use it?

The indexing section of the Google Search Console is used for the (you guessed it) indexing of your website. In this overview, organizations can see the progress of the indexing of a website and the possible reasons why it is or is not going well. In this article, I will elaborate on the possibilities in this section and how I use this section for the technical optimization of websites. (1)

The indexing section outlined

Without getting too deep into it right away, here you can see RalfvanVeen.com’s data from the indexing section of the Google Search Console. In this article I will discuss the sections I use the most myself; pages, sitemaps and deletions. I will discuss each section in more detail. (2)

Overview: pages

Contains 2 sections: reasons for not indexing pages and the progress of indexing pages. Important to note with these sections is that Google is often behind in crawling these pages. The less crawl budget you have as a website, the further Google will fall behind with this.

What is interesting to note is that found – currently not indexed and crawled – currently not indexed is in all likelihood Google’s crawl queue (own interpretation of this).

1. Page redirected – This indicates that the URL has been redirected to another URL. Google may choose to index the target URL instead of the redirected URL. Note that this need not be a problem. It makes sense that a 301 or 302 status code would not be indexed (after all, it is no longer a static HTML page).
2. Alternative page with correct canonical tag – This indicates the existence of another page that Google considers the preferred (canonical) version. The non-indexed page has a “canonical” tag that points to the preferred version that Google should index. In practice, this does often indicate certain problems.
3. Not Found (404) – The server returns a 404 status code, meaning the page cannot be found. These pages are not indexed because Google thinks they don’t exist.
4. Crawled – not currently indexed – Google has crawled the page, but has decided not to include it in the index (yet). This may be temporary or the result of quality problems with the page.
5. Duplicate page, Google has chosen a different canonical page than the user – Google has identified that this page contains duplicate content and has chosen another page as the canonical resource to be indexed. So this could be a problem if this happens more often on the website.
6. Excluded by tag ‘noindex’ – The page contains a ‘noindex’ instruction in the meta tags or headers, which instructs Google not to index this page.
7. Blocked by robots.txt – The page is blocked by a line in the site’s robots.txt file, which instructs crawlers not to crawl the page.
8. Blocked due to other 4xx problem – There is a 4xx client error other than a 404, which may indicate a problem accessing the page, such as a 403 Forbidden or 410 Gone.
9. Found – not currently indexed – Google found the page, but for some reason has not yet indexed it. This may be temporary or dependent on certain factors such as site setup or signals of page quality.
10. Error with redirection – There is a problem with the way the page redirects, possibly due to a misconfigured server or a problem with the redirection chain.

As you can see, these are not all issues. About half of these will end up being issues that can be addressed.

From this data, you can already interpret a general picture of page indexing progress. From this, I can often tell if something is wrong with a website. Important to know: You will almost never get the number of non-indexed pages to 0 here. What Google itself says about this:

These URLs are not indexed by Google. In some cases this may be your intention, but in others it may be a mistake. Review the issues in the table below to determine if you need to modify these URLs.

Read more about the Google Search Console

• The performance report from the Google Search Console
• Verifying your site ownership in the Google Search Console
• Deploy functionalities report within the Google Search Console
• The Google Search Console guide to 2024
• What is the Google Search Console?
• Linking the Google Search Console with Google Analytics
• Add new user Google Search Console

Sitemaps section

In the sitemaps section, you can look more specifically at pages, which means they are all in your XML sitemaps submitted to Google Search Console.(3)

Important to note here is that there are two ways to submit an XML sitemap to the Google Search Console:

1. Robots.txt (you can put a link in the Robots.txt to your website’s XML sitemaps).
2. Google Search Console (via the image above).

One is enough, but I always like to do both anyway to be as explicit as possible in this.

What’s interesting about the sitemaps section is that it allows you to look a lot more specifically at the potential issues of particular pages. When I click through to one XML sitemap:

Then click on view page indexing.

Here you can see the progress again, but very focused on the pages of one particular XML sitemap. What makes it so interesting is that you can now analyze and perform certain optimizations in a much more targeted way. In doing so, you won’t find any “noise” pages here because all these pages are in the XML sitemap.

By these “noise” pages I mean, for example, parameter URLs coming from the faceted navigation of a web shop. Those are also all picked up by GoogleBot and included in the Google Search Console.

For example, if I click here on found – not currently indexed….

Then I end up with a list of URLs that are not indexed for this reason. If I then click through on a specific page, I come out to the summary below.

With this view, I can see in detail why a particular page is not indexed. As can be seen, there is no referring page to this particular page so it is probably not indexed. In the same statement, I can request indexing. This allows you to analyze these URLs at the page level.

Overview: Removals

In this view, as the name suggests, you can quickly remove content from search results.

Three great opportunities.

• Temporary deletions (6 months): This is if you want to quickly remove something that you are likely to remove permanently from the site afterwards.
• Obsolete content: According to Google, this tool only works if the content you want to remove from search results has already been removed from a website.
• Filtering with SafeSearch: This shows all content that has been flagged as explicit by a use in Google in the past six months. Do you feel this is incorrect? Then you can object.(4)

Conclusion

I guess this is not even all that is possible in this section of the Google Search Console, but enough for a first article on this. I use this section mainly for interpretation of the whole thing (how does Google index it) and then the possible dive into specific problems at the URL level. Good luck with optimization!