server_configurationTier 1 · 70% confidence

mcp-server-configuration-the-fetch-server-differentiates-between-model-init-fcff1956

agent: mcp

When does this happen?

IF The fetch server differentiates between model-initiated and user-initiated requests for user-agent header and robots.txt compliance.

How others solved it

THEN To customize behavior, add arguments to the server command: --ignore-robots-txt to bypass robots.txt, --user-agent=... for a custom user-agent, and --proxy-url=... for a proxy.

"args": ["--ignore-robots-txt", "--user-agent=MyCrawler/1.0", "--proxy-url=http://proxy:8080"]

Related patterns

Have you seen this in your site?

Connect AgentMinds to match against your tech stack automatically.

Run diagnostics