NestDaddy
All servers NestDaddy MCP Blog Sign inTry it free
AI and media

evalpair

io.github.Srikarmk/evalpair ยท v0.1.1

Run LLM eval suites on local or hosted models; compare runs with intervals and a noise floor.

Source code Set it up

Connect Runs on your machine

Terminal
claude mcp add evalpair -e EVALPAIR_HOME=<EVALPAIR_HOME> -e EVALPAIR_API_KEY=<EVALPAIR_API_KEY> -- uvx evalpair==0.1.1
claude_desktop_config.json
{
  "mcpServers": {
    "evalpair": {
      "command": "uvx",
      "args": [
        "evalpair==0.1.1"
      ],
      "env": {
        "EVALPAIR_HOME": "<EVALPAIR_HOME>",
        "EVALPAIR_API_KEY": "<EVALPAIR_API_KEY>"
      }
    }
  }
}
~/.cursor/mcp.json
{
  "mcpServers": {
    "evalpair": {
      "command": "uvx",
      "args": [
        "evalpair==0.1.1"
      ],
      "env": {
        "EVALPAIR_HOME": "<EVALPAIR_HOME>",
        "EVALPAIR_API_KEY": "<EVALPAIR_API_KEY>"
      }
    }
  }
}
.vscode/mcp.json
{
  "servers": {
    "evalpair": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "evalpair==0.1.1"
      ],
      "env": {
        "EVALPAIR_HOME": "<EVALPAIR_HOME>",
        "EVALPAIR_API_KEY": "<EVALPAIR_API_KEY>"
      }
    }
  }
}

Replace the values in angle brackets with your own. Keys stay on your machine; we never see them.

Tools

This server runs locally, so its tools are listed by your MCP client once it starts.

Environment variables

NameDescription
EVALPAIR_HOME
Optional		Folder for the call cache, saved runs and your own suites (default ~/.evalpair).
EVALPAIR_API_KEY
Optional, secret		Bearer token for an OpenAI-compatible server, if it needs one (OPENAI_API_KEY also works).

Details

Package[email protected] (PYPI, stdio)
First published5 Nov 2026
Last updated5 Nov 2026
Registry nameio.github.Srikarmk/evalpair

More ai and media servers See all

  • bv7x strategy agents

    Find bv7x prediction agents on Robinhood Chain by market; quote their sealed calls, sold via x402.

  • Chronary

    Calendar API for AI agents: events, availability, Google/Microsoft setup, scheduling, and iCal.

  • Mastering

    Audio mastering for AI agents: LUFS analysis, Suno artifact cleanup, free daily master.

  • Modulos AI Governance Platform

    Evaluate Modulos (EU AI Act, ISO 42001, NIST AI RMF): capabilities, trust, comparisons. Book a demo.

  • naulon

    Check citations and retractions by DOI, and see the x402 price an AI agent pays to cite a source.

  • NextConstraint

    Your SaaS funnel's costliest leak, its fix, watchdog alerts and verdicts, from NextConstraint.

Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.