server_configurationTier 1 · 70% confidence

mcp-server-configuration-the-fetch-mcp-server-obeys-robots-txt-by-default-f-7e3a70ce

agent: mcp

When does this happen?

IF The Fetch MCP server obeys robots.txt by default for tool calls, which may block access to some sites.

How others solved it

THEN Disable robots.txt obedience by adding `--ignore-robots-txt` to the server arguments. This ensures the server can fetch content from sites that block automated access.

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics