NestDaddy

AI Crawler Permission Check

ai.opencurrent/robots-ai-check · v0.1.0

Check a site's robots.txt rules for AI crawlers (GPTBot, ClaudeBot, PerplexityBot and more).

MCP Score How we grade

A server needs 3 daily checks before it gets a grade (1 so far).

Fixed something? Run the same check now. The grade updates at once.

Connect Remote server

Terminal
claude mcp add --transport http robots-ai-check https://opencurrent.ai/mcp/robots
~/.cursor/mcp.json
{
  "mcpServers": {
    "robots-ai-check": {
      "url": "https://opencurrent.ai/mcp/robots"
    }
  }
}
.vscode/mcp.json
{
  "servers": {
    "robots-ai-check": {
      "type": "http",
      "url": "https://opencurrent.ai/mcp/robots"
    }
  }
}
Claude Desktop
Settings > Connectors > Add custom connector
URL: https://opencurrent.ai/mcp/robots

Tools 2 found in our last check

  • check_ai_crawlers

    For a domain (and optional path), whether each known AI crawler/agent user-agent is allowed by robots.txt, with the deciding rule.

  • check_user_agent

    Whether a given crawler user-agent token may fetch a path on a domain according to robots.txt.

Details

Endpointhttps://opencurrent.ai/mcp/robots (Streamable HTTP)
First published21 Nov 2026
Last updated21 Nov 2026
Registry nameai.opencurrent/robots-ai-check

More search and web servers See all

Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.