Why HTTP headers matter for SEO
Every time a browser or a crawler requests a page, the server answers with a status code and a set of headers before the HTML. Some of them change how search engines treat the page, and they are invisible unless you look for them. A forgotten X-Robots-Tag: noindex header can remove a whole site from Google without a single change in the HTML.
What we check
- X-Robots-Tag: the HTTP version of the meta robots tag. It can set noindex, nofollow or nosnippet, and it works for PDFs and images too.
- Link: rel=canonical: a canonical URL sent in the headers, useful for files that have no HTML head.
- Content-Type: the type and character set of the response.
- Content-Encoding: whether the response is compressed with gzip or Brotli, which reduces transfer size.
- Cache-Control: how long browsers and CDNs can store the response.
- Security headers: HSTS, Content-Security-Policy, X-Content-Type-Options, X-Frame-Options, Referrer-Policy and Permissions-Policy.
How to read the results
The tool follows redirects and shows the headers of the final response, so you see what a crawler actually gets. Header names are not case sensitive. If you see a header you do not recognize, it is often added by your hosting provider, CDN or a plugin.
Need to trace the redirects themselves? Use the Redirect Checker.
Frequently asked questions
Is X-Robots-Tag the same as the meta robots tag?
It accepts the same directives. Search engines combine both, and the most restrictive rule applies.
Do security headers affect rankings?
Not directly, apart from HTTPS itself. They protect your visitors and are a sign of a well-maintained site.
Why are the headers different from what I see in my browser?
CDNs and servers can answer differently depending on location, cookies, user agent or cache state.