GuidesIndexed

Excluded by ‘noindex’ tag: where is the noindex?

You open the page source, search for noindex and find nothing. Search Console still lists the page under Excluded by ‘noindex’ tag. Believe the report anyway. The noindex may be somewhere view-source can’t show you.

Josh Pigford, founder of FirstIndex6 min read
The short answer

Read the response headers, not just the page source. This row means Google crawled the page and found a noindex in one of two places: a meta tag in the HTML, or an X-Robots-Tag header that view-source never shows. Remove it where it’s set, then run the live test in URL Inspection.

Did you mean to hide this page?

Then keep the noindex and take the page out of your sitemap. Thank-you, trial, demo, booking and legal pages are the usual ones, and Google says it’s fine for a URL not to be indexed “for the right reasons,” a noindex tag among them. Our own 404 page and the page you see after buying our Search and AI report both carry one, and neither is in our sitemap. Google’s help lists a sitemap with noindex URLs in it as one cause of a spike in this report.

On November 14, 2026, we checked 291 startup sites, a random sample of active B2B companies from a public copy of Y Combinator’s directory. 240 had a sitemap we could read, and 10 of those listed a few pages that carried a noindex tag. To find yours, pick the All submitted pages filter at the top of the report: it shows only the rows that came from a sitemap.

Don’t swap the noindex for a robots.txt block, either. Blocked, Google never reads the noindex, and the page “can still appear in search results” if other pages link to it. URL Inspection won’t warn you: with the page blocked, Indexing allowed? “will always be "Yes".” Our robots.txt checker shows whether a rule blocks a URL.

Why can’t you find the noindex in the page source?

Because a noindex can live in two places. One is a meta tag in the HTML, <meta name="robots" content="noindex">, and view-source shows that. The other is an X-Robots-Tag: noindex header on the response, which never appears in the page at all. Google reads both, and a tag that says none or one addressed to googlebot counts too.

To see the headers, request the page and print them:

Terminal · from curl’s manual

curl -s -D - -o /dev/null https://example.com/pricing | grep -i x-robots-tag

-D - prints the response headers, and -o /dev/null throws the page itself away. If the command prints a line, that’s your noindex. URL Inspection names the place too. For a header, it reads “Indexing allowed: No, 'noindex' detected in 'X-Robots-Tag' HTTP header.”

Where a noindex header shows up, and where it doesn’tAn example for ExampleCo’s pricing page. In a terminal, curl prints the response headers, and one of them is x-robots-tag: noindex. In the browser’s view-source window, the same page’s HTML head has a title, a description and a canonical link, and a search for noindex finds 0 matches. Terminal $ curl -s -D - -o /dev/null \https://example.com/pricingHTTP/2 200content-type: text/html x-robots-tag: noindex View source Find “noindex”: 0 matches <head><title>Pricing · ExampleCo</title><meta name="description" …><link rel="canonical" …></head>
The same ExampleCo page in a terminal and in view-source. ExampleCo is an example company.

The header is also where the worst cases hid in our test. On 2 of the 240 sitemaps, every URL we checked said noindex, and both times it was the header. Of the 291 homepages, 2 carried a noindex: one header, on one of those two sites, and one tag.

Startup sitemaps that list pages saying noindexFirstIndex test, November 14, 2026, 240 startup sitemaps. 10 listed a few pages with a noindex tag in the HTML. 2 listed only pages that sent a noindex header. The other 228 listed none among the URLs checked. 10 of 240 list a few pages with a noindex tag 2 of 240 list only pages with a noindex header
Each square is one startup sitemap: the first 25 URLs on each, so these are lower bounds. FirstIndex test, November 14, 2026, 240 sitemaps we could read out of 291 startup sites.

What adds a noindex you didn’t write?

Usually your host, your site builder or your framework. Hosts add a header to their preview addresses, so Google may have been sent a preview address, not your real page. Site builders and frameworks like Next.js write a tag from a setting:

Where it comes fromLands asWhen
VercelHeaderOn preview deployments, and on a production deployment opened at its own deployment URL
NetlifyHeaderOn Deploy Previews
Cloudflare PagesHeaderOn every preview deployment, by default
WordPressTagWhen “Discourage search engines from indexing this site” is ticked in Settings › Reading
WebflowTagFrom a page’s Sitemap indexing toggle, or a CMS item with indexing turned off
Next.jsTagFrom robots in the metadata: set in a layout, it reaches every page below it that doesn’t set its own

From each one’s docs: Vercel, Netlify, Cloudflare Pages, WordPress, Webflow and Next.js.

One of the two startups whose sitemap said noindex all the way down shows how a preview header reaches Google. Its sitemap listed its .vercel.app deployment addresses instead of its own domain. Those addresses send Vercel’s noindex header, so every URL the sitemap handed Google said noindex, while its homepage on its own domain didn’t. The fix is a sitemap built from your production domain.

The other one sent the header from its own domain, on every page we checked, homepage included. Vercel doesn’t list a production domain among the places it adds one, so that header was the site’s own. Look for a header rule in your host’s settings and your app’s config.

Why does Search Console still say noindex after you removed it?

Because the report shows what Google saw on its last crawl, not what your site sends now. Open the URL in URL Inspection and run the live test. If Indexing allowed? no longer finds a noindex, click Request indexing.

If the live test still finds one, check two things. A CDN or cache may still be serving the old page: Google’s John Mueller has seen this report flag a noindex “with CDNs.” Or the noindex is in the HTML your server sends and JavaScript removes it in the browser. When Google meets a noindex, it “may skip rendering and JavaScript execution,” so take it out of the original page code.

Once the pages you fixed pass the live test, click Validate fix on the row. Google says validation “typically takes up to about two weeks, but in some cases can take much longer.”

Other rows in the same report: Soft 404, Page with redirect, or the whole Page indexing report at once.

A noindex nobody meant to ship is what we catch in our plan: we check that Google and AI crawlers can find, read and keep every page. It’s $2,000 a month for software companies. See what’s in the plan.

Questions

What does non-indexed mean?

That Google knows the URL but hasn’t added the page to its index, so it can’t show in Google’s results. It can’t be a link in AI Overviews or AI Mode either: Google says a page “must be indexed” to appear there. The Page indexing report lists every reason a page isn’t.

Does a noindex page still pass links?

For a while. In 2017, Google’s John Mueller said that once a page has carried noindex long enough, Google removes it completely, “and then we won’t follow the links anyway.” So don’t rely on a noindexed page’s links to get other pages found.

Is it bad to have lots of noindex pages?

No. Google’s Gary Illyes said in 2022 that “having many pages with noindex will not influence how Google crawls and indexes your site.”