Noindex
Also known as: noindex tag, noindex directive
A directive, set via a meta robots tag or an HTTP header, that tells search engines to keep a specific page out of their index so it never appears in results. The page can still be crawled and its links followed, but it will not surface for any query.
Key Takeaways
- Noindex is a directive, set via a meta robots tag or HTTP header, that keeps a page out of search results.
- A noindexed page can still be crawled and its links followed, but it will not appear for any query.
- It gives precise control over which pages compete in search, useful for thin, duplicate, or private pages.
- Blocking a noindexed page in robots.txt prevents crawlers from seeing the directive, so the page may stay indexed.
- Leaving noindex on after a staging launch will deindex live pages that should be visible.
How It Works
Noindex is a signal you place in a page's meta robots tag or in an HTTP response header. When a crawler fetches the page and reads the directive, it removes that URL from the index or keeps it out, so the page never surfaces in results. The page remains fully accessible to users, and crawlers can still follow its links unless you also add nofollow.
The common failure is combining noindex with a block in Robots.txt. If robots.txt disallows the URL, the crawler never fetches the page, so it never reads the noindex directive, and the page can linger in the index from prior discovery.
Noindex often works alongside a Canonical Tag to manage duplication, though they solve different problems. You can confirm which pages are excluded using the Index Coverage report, which lists URLs kept out by noindex.
Why It Matters
Noindex gives precise control over which pages compete in search, keeping thin, duplicate, or private pages like thank-you screens and internal search results out of the index while preserving them for users.
Example
A SaaS site noindexes its thank-you page, internal search results, and login screen so they never clutter search results. During a redesign, a developer accidentally leaves the site-wide noindex tag from staging on the live site. Within days, real pages start dropping from results. Removing the leftover tag and requesting reindexing restores the pages once Google recrawls them.
Common Mistake
Blocking a noindexed page in robots.txt as well, which stops crawlers from ever seeing the noindex directive, so the page can linger in the index. Leaving noindex on after a staging launch also deindexes live pages.
Frequently Asked Questions
What is the difference between noindex and robots.txt?
Noindex keeps a page out of the index but still lets crawlers fetch it. Robots.txt blocks crawling entirely. If you block a page in robots.txt, crawlers cannot see a noindex directive, so it may stay indexed.
Does noindex stop a page from being crawled?
No. A noindexed page can still be crawled, and its links can still be followed. Noindex only keeps the page out of search results, so it will not appear for any query.
When should I use noindex?
Use noindex for pages that should exist for users but not compete in search, such as thank-you pages, internal search results, login screens, and thin or duplicate pages you want to keep private from results.