H6 · Infrastructure, Bots & DNS

Security Headers Checker — are your HTTP headers configured?

**HTTP response headers are a quiet but powerful layer — they can harden your site against attacks and control how engines index it, all before any HTML loads.** This check looks at your security headers (like HSTS and Content-Security-Policy) and robots-related headers (like X-Robots-Tag). Sensible headers improve security and give you server-level control over indexing that the page markup can't.

What does the headers check look for?

It inspects your HTTP response headers for both security and indexing controls. Specifically:

- HSTSStrict-Transport-Security forcing browsers to use HTTPS.

- Content-Security-Policy — limiting what resources can load, reducing injection risk.

- Other protections — headers like X-Content-Type-Options and Referrer-Policy.

- X-Robots-Tag — server-level indexing directives, useful for non-HTML resources like PDFs.

Sensible security and robots headers passes; some present but key ones missing is a warning; no meaningful security headers is a fail.

How is it evaluated, and how is it scored?

GEObubbly inspects the page's HTTP response headers for security and robots directives. It's an extended Infrastructure check that runs partially, since it reads the live response headers.

Criteria: Pass — reasonable security headers. Warning — missing some security headers. Fail — no security headers.

Why security and robots headers matter for SEO and GEO

HTTP response headers are instructions the server sends with every response, before any HTML — and they do two useful jobs. On the security side, headers like Strict-Transport-Security (HSTS, which forces browsers to always use HTTPS), Content-Security-Policy (which restricts what resources a page can load), X-Content-Type-Options and Referrer-Policy harden your site against common attacks. A well-secured site is part of the trust baseline engines and users expect. On the indexing side, the X-Robots-Tag header lets you apply directives like noindex at the server level — crucially, this works for non-HTML resources like PDFs and images where you can't add a meta robots tag, giving you control the page markup can't. Together, good headers are a low-visibility, high-value configuration layer: they don't change what users see but meaningfully improve security posture and indexing control.

How this check scores

  • Pass: Reasonable security headers and robots controls in place.
  • Warning: Some headers present, key ones (HSTS or CSP) missing.
  • Fail: No meaningful security headers configured.

FAQ

What are HTTP security headers?

HTTP security headers are directives a server includes in its responses to instruct the browser to behave more securely. Common ones include Strict-Transport-Security (HSTS), which forces browsers to use HTTPS and not fall back to insecure HTTP; Content-Security-Policy (CSP), which restricts what scripts and resources a page can load to mitigate cross-site scripting; X-Content-Type-Options, which stops browsers guessing content types; and Referrer-Policy, which controls referrer information. Together they harden your site against several classes of attack at the browser level, complementing HTTPS and clean code.

What is HSTS and why is it useful?

HSTS (Strict-Transport-Security) is a security header that tells browsers to only ever connect to your site over HTTPS, even if a user types or clicks an http:// link. Once a browser has seen the HSTS header, it automatically upgrades all requests to your domain to HTTPS for the specified duration, preventing downgrade attacks and accidental insecure connections. It's a strong complement to having HTTPS: where HTTPS secures the connection, HSTS guarantees the secure connection is always used. Enabling it is a recommended step for any site already on HTTPS, though it should be configured carefully since it's enforced strictly.

What does the X-Robots-Tag header do?

The X-Robots-Tag HTTP header applies robots directives — like noindex, nofollow or noarchive — at the server response level, rather than in the page's HTML. Its key advantage is that it works for non-HTML resources where you can't insert a meta robots tag, such as PDFs, images and other files. So if you need to keep a PDF out of the index, an X-Robots-Tag: noindex header on that file does the job. It gives you server-level control over indexing across all resource types, complementing the meta robots tag you'd use on HTML pages.

Do security headers affect SEO?

Indirectly. Security headers aren't direct ranking factors, but they harden your site and contribute to the trust and security baseline that a well-built, credible site demonstrates — and HSTS in particular reinforces the HTTPS that is a ranking signal. A secure, properly-configured site is less likely to be compromised (which would harm SEO badly) and presents better to users and engines. The robots-related X-Robots-Tag header does have a direct SEO role, since it controls indexing. Overall, good headers support the security and indexing control that underpin a healthy SEO foundation.

Do HTTP headers matter for AI crawling?

Yes, in a couple of ways. The X-Robots-Tag header controls indexing directives that crawlers respect, including for non-HTML resources, so it shapes what's available to be indexed and potentially cited. And security headers contribute to the overall picture of a hardened, trustworthy, well-maintained site, which is the kind of source AI engines favour. While most security headers aren't a direct GEO factor, a properly-configured header layer signals a professionally-run site and gives you precise control over what crawlers can index, both of which support your standing for AI visibility.

Audit your page across all 9 checks in Infrastructure, Bots & DNS

Run a free audit →