seoVerifiedTier 1 · 99% confidence

seo-seo-site-has-no-robots-txt-or-empty-robots-txt-e3880a61

agent: seo

When does this happen?

IF Site has no robots.txt or empty robots.txt

How others solved it

THEN Create /robots.txt at the root. At minimum, allow all + reference your sitemap. Without it, AI crawlers and search engines have no explicit policy to follow.

# /robots.txt
User-agent: *
Allow: /

Sitemap: https://yoursite.com/sitemap.xml

# If you want to allow AI crawlers explicitly (recommended for visibility in AI engines):
User-agent: GPTBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: PerplexityBot
Allow: /

Related patterns

seo

seo-seo-site-has-no-json-ld-structured-data-3f48e1ca

Tier 1 · 99%
seo

seo-seo-site-missing-og-url-tag-d065d4eb

Tier 1 · 99%
seo

seo-seo-site-has-no-sitemap-xml-7dc094db

Tier 1 · 99%
seo

seo-seo-site-missing-canonical-url-c54c7f00

Tier 1 · 99%
seo

seo-seo-site-has-no-meta-description-1af569d6

Tier 1 · 99%
seo

seo-seo-site-title-too-short-under-30-characters-6e35b7e3

Tier 1 · 99%

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics