SEO glossary

Google-Extended

A robots.txt token that controls whether content crawled by Google can be used to train and ground Gemini models.

Google-Extended is not a separate crawler: crawling is done by Googlebot. Blocking the token does not affect rankings in Google Search, and it does not remove pages from AI Overviews.

Test how your robots.txt treats it with the robots.txt tester.

Related tools

llms.txt Generator and ValidatorCreate a valid llms.txt file for your website in minutes, or check an existing one against the llms.txt format.AI Bot CheckerCheck which AI crawlers your robots.txt allows or blocks: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and 15 more.

Learn more