Claude-User
Fetches a page live when a Claude user’s request references it.
| Operator | Anthropic |
| Role | Live answer fetcherFetches your pages in real time while an AI model answers a question. The strongest signal AI is reading you live. |
| User agent token | Claude-User |
| robots.txt | The operator documents honouring robots.txt. |
| Operator docs | support.anthropic.com |
Allow or block Claude-User in robots.txt
Add either rule to the robots.txt at your site root. robots.txt is a request, not a lock: this operator documents honouring it, so the rule is normally enough.
Allow
User-agent: Claude-User Allow: /
Block
User-agent: Claude-User Disallow: /
Worth knowing before you block: this is a retrieval crawler. Blocking it removes your pages from what the AI models can read and cite live, which usually costs visibility rather than protecting anything.
Other Anthropic crawlers
- Claude-SearchBotAI search crawler
- ClaudeBotTraining crawler
- Anthropic-AIOn-demand fetcher
Is Claude-User allowed on your site right now?
The free checker reads your robots.txt and probes your site as each AI crawler, so you see the policy and the reality in one pass.