Google’s New Spam Advice: Only Index Content Meant for Search

Google's New Spam Advice: Only Index Content Meant for Search

Written by

Mauricio Rodriguez

in

On October 26, 2026, Google updated its guide to preventing user-generated spam with a new section titled “Only index content meant for Search”. It is aimed at a growing problem: tools that let anyone publish a page, from document sharing to AI generators and app builders, being used to mass-produce spam pages on trusted domains.

What changed

  • Wider scope. The guide now covers “tools that let users publish shared pages, generated content, or web apps”, and warns that spammers may create many “pages or sites”, not only sites.
  • Noindex by default. Pages made for sharing, collaboration or utility should be kept out of the index by default, with a noindex tag or X-Robots-Tag header, or blocked from crawling.
  • Monitoring. Platforms should verify their site or subdomain in Search Console to receive manual action and security alerts, and watch for unexpected spikes in indexed pages, impressions or queries.
  • Platform metrics. Google recommends tracking page creation and indexing internally, with alerts when new public pages spike or a few users create a large share of indexable URLs.
  • Unchanged: the earlier advice to noindex posts from new users until they build a reputation.

Who is affected

  • Platforms that host user pages: document and file sharing, form builders, link-in-bio pages, AI content and app generators, website builders on shared domains.
  • SEOs who use those platforms for links or visibility. Publishing pages on a trusted domain to borrow its authority, including for AI search visibility, is exactly what this guidance targets. Search Engine Journal notes the update does not prove a crackdown is coming, but it shows where Google is looking.

The robots.txt trap

The most useful part of the update is a reminder about how blocking works:

  • robots.txt does not keep a URL out of the index. A blocked URL can still be indexed, without a description, if other sites link to it. In Search Console it appears as “Indexed, though blocked by robots.txt”.
  • noindex only works if Google can crawl the page. If robots.txt blocks it, Googlebot never sees the noindex.
  • So for shared and generated pages, use noindex and allow crawling. Use robots.txt only when you need to save crawling, not to hide pages.
<!-- in the page head -->
<meta name="robots" content="noindex">

# or as an HTTP header, also valid for PDFs and files
X-Robots-Tag: noindex

Check whether a URL is blocked or noindexed with our robots.txt tester and HTTP Header Checker, and audit the meta robots tag of any page with the SEO Audit Tool. For background, read our robots.txt guide.

Frequently asked questions

Frequently asked questions

What did Google change in its spam guidance?

On October 26, 2026, it added advice for platforms that host shared pages, generated content or web apps: block indexing of those pages by default and monitor for spikes in indexed pages.

Does robots.txt prevent a page from being indexed?

No. A URL blocked by robots.txt can still be indexed if other sites link to it. Use a noindex tag and let Google crawl the page to see it.

Is Google cracking down on parasite SEO on platforms?

The guidance shows Google is paying closer attention to abuse of publishing platforms. It does not announce a specific crackdown.

How do I noindex a PDF or file?

Send an X-Robots-Tag: noindex HTTP header with the file, since PDFs cannot carry a meta robots tag.

More posts