A page can be blocked from search results two different ways: a <meta name="robots" content="noindex"> tag in the HTML, or an X-Robots-Tag HTTP response header set by the server — invisible unless you look at the raw response, since it never appears in the page’s source. Either one alone is enough to keep a page out of search results, which is exactly why checking only the page’s HTML misses half the picture.

Why the header exists at all

The meta tag only works on HTML. A PDF, an image, or any non-HTML file has nowhere to put a <meta> tag — the X-Robots-Tag header does the identical job for any content type, set at the server or CDN level instead of in a template. It also means a noindex can be applied to an entire path pattern (every file under /private/, for instance) from one server config rule, without touching a single template.

What each directive means

• noindex — don’t show this page in search results.
• nofollow — don’t pass ranking signal through links on this page.
• none — shorthand for noindex, nofollow together.
• noarchive — don’t keep a cached copy.
• nosnippet — don’t show a text preview or video snippet in results.
• max-snippet:N — cap the preview text to N characters (0 disables it, -1 means no limit).

Why the checker looks at both, combined

The stricter directive from either source wins — a page with index in its meta tag but noindex in the header is still blocked. This combined view is exactly what catches the common accident: a CDN or reverse-proxy rule set to noindex an entire staging subdomain, left in place after the subdomain became the production site, with the HTML itself never mentioning noindex anywhere.

How this differs from the SEO Optimization Checker

The SEO Optimization Checker includes an indexability check as one signal among many across the whole page. This tool does one thing — both robots signals, combined, with the exact directive text from each source — for when indexability itself is the question.

Frequently asked

Which wins if the meta tag says index but the header says noindex?

The header. Google explicitly documents that when directives conflict, the most restrictive one applies, regardless of which source it came from.

Can robots.txt cause the same problem?

robots.txt controls crawling, not indexing — a blocked-from-crawling page can still be indexed (with no snippet) if enough links point to it. noindex is the only reliable way to keep a page out of results entirely. Check crawl access separately with the Robots.txt Tester.

Does this tool send my page anywhere?

No — one request goes to the page itself to read its headers and HTML; everything else runs on that response.