Weekly ecommerce tips, deals & news.
Indexability is whether a search engine is allowed and able to store your page in its index. A page can be perfectly visible to shoppers and still be invisible to Google. If it never enters the index, it can never rank for anything. For an online store, that usually decides whether a product page earns traffic or sits there silently.
Picture a librarian walking the aisles of a warehouse. Crawling is the walk. Indexing is the moment they write a book onto a card and file it in the catalogue.
A book can be picked up and still never filed. That is the gap where most store pages get lost.
Google describes its process in three stages. Each one is a separate gate your page has to pass.
Google is blunt about the odds. Its documentation states it does not guarantee it will crawl, index, or serve your page. That holds even when the page follows every guideline.
The scale of that filtering is easy to underestimate. Ahrefs studied around 14 billion pages and found 96.55% get no traffic from Google. Indexability is the first hurdle in that number.
Some blocks are deliberate and some are accidents. Both look identical from the outside.
WordPress adds one more. A single checkbox under Reading settings discourages search engines site-wide, and it survives long after a site goes live. Always check it first when a whole store vanishes.
Then there are the technical accidents. A page returning a server error cannot be indexed, and neither can one buried behind a long chain of redirects.
Slow pages suffer quietly too. If a page times out often enough, Googlebot simply visits less, so new products take longer to appear.
Orphan pages are the last common cause. A product with no internal link pointing at it may never be discovered, even though nothing blocks it. Therefore your category structure is an indexing tool, not just navigation.
This is the mistake that catches experienced people. Blocking a page in robots.txt does not remove it from search results.
Google’s documentation is explicit. For a noindex rule to work, the page must not be blocked by robots.txt. It also has to stay reachable by the crawler.
The logic is simple once you see it. Googlebot has to read the page to find the noindex instruction. Block the crawl and the instruction is never delivered.
So a blocked page can still appear in results, especially if other sites link to it. The fix is counterintuitive: allow the crawl, and let the noindex do its job. Our robots.txt entry covers the file itself in more detail.
Indexing everything is not the goal. A store generates a lot of URLs that add nothing to search.
Filtered URLs are the biggest offender in e-commerce. A shop with five filters can generate thousands of near-identical addresses. Each one consumes crawl budget that your product pages needed.
Guessing is not necessary here. Two checks tell you almost everything, and both are free.
Compare the indexed count against your real product count. A store with 3,000 products and 40,000 indexed URLs has a problem, even if traffic looks fine.
The gap in the other direction matters more. Fewer indexed pages than products means genuine inventory is missing from search entirely.
Getting indexed only buys a ticket, of course. Ahrefs puts 96.6% of all clicks on page one results, so an indexed page ranking at number forty earns nothing.
Imagine a bike parts store called Cogsworth Cycles. They stock 3,000 products with filters for size, colour, brand, and material.
Traffic has been flat for months despite steady publishing. A coverage report shows roughly 40,000 URLs known to Google, against 3,000 real products.
Worse, many genuine product pages are marked as discovered but not indexed. Meanwhile filtered URLs are being crawled repeatedly.
The filters are the cause. Every combination of size and colour creates a new address, so 3,000 products bloom into tens of thousands of URLs.
Somebody had tried to fix this already. They added a robots.txt disallow for the filter parameter, which stopped the crawl but left the URLs in search results.
That is the trap in action. The pages were blocked rather than removed, so Google kept listing them without ever reading the noindex sitting inside.
Cogsworth reverse the order. First they remove the robots.txt disallow, letting Googlebot read the filtered pages again.
Then they apply noindex to filtered URLs and point canonicals at the clean category page. Google reads the instruction and drops them over the following weeks.
Their XML sitemap is trimmed to the 3,000 products and the main categories. Nothing else belongs in it.
The result is not instant, and that matters for expectations. Ahrefs found only 1.74% of new pages reach the top ten within a year. Getting indexed is the start of the process, not the finish.
Crawlability is whether a search engine can reach and read a page. Indexability is whether it is allowed to store it afterwards.
The order matters, because crawlability comes first. A page that cannot be crawled cannot be properly indexed, since nothing was read to index.
The reverse is not true, though. A page can be perfectly crawlable and still excluded, either by your own noindex or by Google’s judgement.
In practice you diagnose them in sequence. Ask whether Google can reach the page, then ask whether anything tells it not to keep the page. For marketplaces with many vendor pages, our guide to SEO for marketplaces covers how that scales.
Usually Google read the page and decided it was not worth storing. That normally points at thin content, near-duplicate text, or a page that repeats an existing one.
Product pages using only the manufacturer’s description are a classic case. Rewriting them with original detail is a more reliable fix than resubmitting the URL.
Add a noindex rule and make sure the page is still crawlable so Google can see it. For anything urgent, use the removal tool in Search Console as well.
The removal tool is temporary, so treat it as a stopgap. The noindex is what makes the change permanent once Google recrawls.
No, and a smaller index is often healthier. What matters is the share of your indexed pages that are genuinely useful.
Thousands of thin filtered URLs dilute how a search engine reads your site. Cutting them back concentrates attention on the pages you actually want ranking.
Indexability decides whether your pages are even eligible to rank. Allow crawls where it counts, use noindex deliberately, and keep junk URLs out of your sitemap. A smaller, cleaner index nearly always beats a large messy one.
Copyright © StoreOwnerTips.com. All Rights Reserved.
