ClaudeBot
Crawls public content to train Anthropic’s Claude models.
| Operator | Anthropic |
| Role | Training crawlerCrawls public content to train future AI models. Blocking it is a policy choice, not a visibility fix. |
| User agent token | ClaudeBot |
| robots.txt | The operator documents honouring robots.txt. |
| Operator docs | support.anthropic.com |
Allow or block ClaudeBot in robots.txt
Add either rule to the robots.txt at your site root. robots.txt is a request, not a lock: this operator documents honouring it, so the rule is normally enough.
Allow
User-agent: ClaudeBot Allow: /
Block
User-agent: ClaudeBot Disallow: /
Other Anthropic crawlers
- Claude-UserLive answer fetcher
- Claude-SearchBotAI search crawler
- Anthropic-AIOn-demand fetcher
Is ClaudeBot allowed on your site right now?
The free checker reads your robots.txt and probes your site as each AI crawler, so you see the policy and the reality in one pass.