Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).
MCP Score How we grade
A server needs 3 daily checks before it gets a grade (1 so far).
Connect Remote server
claude mcp add --transport http robots-ai-check https://opencurrent.ai/mcp/robots
{
"mcpServers": {
"robots-ai-check": {
"url": "https://opencurrent.ai/mcp/robots"
}
}
}
{
"servers": {
"robots-ai-check": {
"type": "http",
"url": "https://opencurrent.ai/mcp/robots"
}
}
}
Settings > Connectors > Add custom connector
URL: https://opencurrent.ai/mcp/robots
Tools 2 found in our last check
check_ai_crawlersFor a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule.
check_user_agentWhether a given crawler user-agent token may fetch a path on a domain according to robots.txt.
Details
| Endpoint | https://opencurrent.ai/mcp/robots (Streamable HTTP) |
| First published | 21 Nov 2026 |
| Last updated | 21 Nov 2026 |
| Registry name | ai.opencurrent/robots-ai-check |
More search and web servers See all
Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.
