NestDaddy

AI Crawlability Check

io.github.Milan344s/ai-crawlability-check · v2.0.0

Check if AI crawlers like GPTBot and ClaudeBot can reach, index and quote any web page.

MCP Score How we grade

A+
99 / 100

From what our daily checks can see without credentials, last checked 21 Nov 2026.

Reliability
Answered 3 of 3 daily checks.
30 / 30
Speed
Typical response 104 ms.
20 / 20
Tool docs
5 of 5 tools have a full description; 6 of 6 inputs have a description.
25 / 25
Stability
No tool rewrites in 30 days.
15 / 15
Listing
Missing an icon.
9 / 10

How to raise the grade

  • Add an icon to your server.json in the official MCP registry.
Fixed something? Run the same check now. The grade updates at once.

Connect Remote server

Terminal
claude mcp add --transport http ai-crawlability-check https://mcp.milan-novotny.com/mcp
~/.cursor/mcp.json
{
  "mcpServers": {
    "ai-crawlability-check": {
      "url": "https://mcp.milan-novotny.com/mcp"
    }
  }
}
.vscode/mcp.json
{
  "servers": {
    "ai-crawlability-check": {
      "type": "http",
      "url": "https://mcp.milan-novotny.com/mcp"
    }
  }
}
Claude Desktop
Settings > Connectors > Add custom connector
URL: https://mcp.milan-novotny.com/mcp

Tools 5 found in our last check

  • check_ai_crawler_access

    Reports which AI crawlers may fetch a public URL. Reads the site's robots.txt and returns allowed or blocked, with the deciding rule, for each known AI crawler from OpenAI, Anthropic, Perplexity, Google, Microsoft, Apple, Meta, Amazon and others, split into search and answer-tim…

  • check_ai_indexability

    Checks whether a public page can be indexed by search and AI engines once a crawler reaches it. Measures HTTP status and redirects, noindex, canonical target, how much copy exists in the raw HTML without JavaScript, client-side rendering signals, title and H1, XML sitemap covera…

  • check_ai_citability

    Fetches a public page and scores how quotable its content is for AI engines once it is crawled and indexed. Measures answer-first structure, question headings, lists and tables, FAQ, paragraph and sentence length, statistic and source density, named entities, schema, author, fre…

  • check_seo_basics

    Fetches a public page and runs 24 measurable on-page SEO checks: title and meta description length, H1, heading hierarchy, keyword placement and density, slug, canonical, internal and external links, anchor text, image alt text, schema and social tags. Pass the target keyword to…

  • get_review_rubric

    Returns a scoring rubric as text for the judgment items the URL tools cannot measure: geo (would an AI engine quote this article?) or seo (pre-publish SEO review). Fetches nothing.

Details

Endpointhttps://mcp.milan-novotny.com/mcp (Streamable HTTP)
First published19 Nov 2026
Last updated19 Nov 2026
Registry nameio.github.Milan344s/ai-crawlability-check

More search and web servers See all

Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.