NestDaddy
All servers NestDaddy MCP Blog Sign inTry it free
Security

Evals Mcp Server

io.github.cyanheads/evals-mcp-server · v0.1.6

Author verifiable eval records through a draft→review→revise→submit loop with enforced graders.

Source code Set it up

Connect Runs on your machine

Terminal
claude mcp add evals-mcp-server -e EVALS_DATA_DIR=<EVALS_DATA_DIR> -e EVALS_REQUIRE_CONFIRMATION=<EVALS_REQUIRE_CONFIRMATION> -e EVALS_DEFAULT_LICENSE=<EVALS_DEFAULT_LICENSE> -e EVALS_CAPTURE_DIR=<EVALS_CAPTURE_DIR> -e MCP_LOG_LEVEL=<MCP_LOG_LEVEL> -- npx -y @cyanheads/[email protected]
claude_desktop_config.json
{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/[email protected]"
      ],
      "env": {
        "EVALS_DATA_DIR": "<EVALS_DATA_DIR>",
        "EVALS_REQUIRE_CONFIRMATION": "<EVALS_REQUIRE_CONFIRMATION>",
        "EVALS_DEFAULT_LICENSE": "<EVALS_DEFAULT_LICENSE>",
        "EVALS_CAPTURE_DIR": "<EVALS_CAPTURE_DIR>",
        "MCP_LOG_LEVEL": "<MCP_LOG_LEVEL>"
      }
    }
  }
}
~/.cursor/mcp.json
{
  "mcpServers": {
    "evals-mcp-server": {
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/[email protected]"
      ],
      "env": {
        "EVALS_DATA_DIR": "<EVALS_DATA_DIR>",
        "EVALS_REQUIRE_CONFIRMATION": "<EVALS_REQUIRE_CONFIRMATION>",
        "EVALS_DEFAULT_LICENSE": "<EVALS_DEFAULT_LICENSE>",
        "EVALS_CAPTURE_DIR": "<EVALS_CAPTURE_DIR>",
        "MCP_LOG_LEVEL": "<MCP_LOG_LEVEL>"
      }
    }
  }
}
.vscode/mcp.json
{
  "servers": {
    "evals-mcp-server": {
      "type": "stdio",
      "command": "npx",
      "args": [
        "-y",
        "@cyanheads/[email protected]"
      ],
      "env": {
        "EVALS_DATA_DIR": "<EVALS_DATA_DIR>",
        "EVALS_REQUIRE_CONFIRMATION": "<EVALS_REQUIRE_CONFIRMATION>",
        "EVALS_DEFAULT_LICENSE": "<EVALS_DEFAULT_LICENSE>",
        "EVALS_CAPTURE_DIR": "<EVALS_CAPTURE_DIR>",
        "MCP_LOG_LEVEL": "<MCP_LOG_LEVEL>"
      }
    }
  }
}

Replace the values in angle brackets with your own. Keys stay on your machine; we never see them.

Tools

This server runs locally, so its tools are listed by your MCP client once it starts.

Environment variables

NameDescription
EVALS_DATA_DIR
Optional		Root folder for record JSON. The store manages drafts/, submitted/, and exports/ subdirs under it.
EVALS_REQUIRE_CONFIRMATION
Optional		When 'true', evals_submit_draft requests human confirmation before finalizing.
EVALS_DEFAULT_LICENSE
Optional		Default metadata.license applied when a draft omits one.
EVALS_CAPTURE_DIR
Optional		Directory of framework-written tool-call captures; when set, captures EvalsIDs resolve to full dumps.
MCP_LOG_LEVEL
Optional		Sets the minimum log level for output (e.g., 'debug', 'info', 'warn').

Details

Package@cyanheads/[email protected] (NPM, stdio)
Package@cyanheads/[email protected] (NPM, streamable-http)
First published13 Oct 2026
Last updated13 Oct 2026
Registry nameio.github.cyanheads/evals-mcp-server

More security servers See all

Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.