EvalForge Lite

io.github.thejaredchapman/evalforge-lite · v1.0.1

Compare text LLMs across OpenRouter, Bedrock, Vertex AI and Foundry with automated grading.

Website Source code Set it up

Connect Runs on your machine

Terminal

claude mcp add evalforge-lite -e OPENROUTER_API_KEY=<OPENROUTER_API_KEY> -- uvx evalforge-lite==1.0.1

claude_desktop_config.json

{
  "mcpServers": {
    "evalforge-lite": {
      "command": "uvx",
      "args": [
        "evalforge-lite==1.0.1"
      ],
      "env": {
        "OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
      }
    }
  }
}

~/.cursor/mcp.json

{
  "mcpServers": {
    "evalforge-lite": {
      "command": "uvx",
      "args": [
        "evalforge-lite==1.0.1"
      ],
      "env": {
        "OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
      }
    }
  }
}

.vscode/mcp.json

{
  "servers": {
    "evalforge-lite": {
      "type": "stdio",
      "command": "uvx",
      "args": [
        "evalforge-lite==1.0.1"
      ],
      "env": {
        "OPENROUTER_API_KEY": "<OPENROUTER_API_KEY>"
      }
    }
  }
}

Replace the values in angle brackets with your own. Keys stay on your machine; we never see them.

Tools

This server runs locally, so its tools are listed by your MCP client once it starts.

Environment variables

Name	Description
`OPENROUTER_API_KEY` Optional, secret	Optional. OpenRouter API key held by the server so tool calls need no credentials for OpenRouter. If unset, pass creds in each tool call.

Details

Package	`[email protected]` (PYPI, stdio)
First published	1 Oct 2026
Last updated	1 Oct 2026
Registry name	`io.github.thejaredchapman/evalforge-lite`

More other servers See all

Live status

Local install

Installed and run on your own machine, so there is no public endpoint to check.

Pair it with live search

Agents that use EvalForge Lite often also need current facts. NestDaddy MCP gives your AI today's internet: web search, real-time breaking news and full page reading in one connector.

See NestDaddy MCP

For maintainers

[![NestDaddy MCP](https://nestdaddy.com/mcp/servers/io-github-thejaredchapman-evalforge-lite/badge.svg)](https://nestdaddy.com/mcp/servers/io-github-thejaredchapman-evalforge-lite)

Add the badge to your README. It turns green while your server passes our daily check. Listing details come from the official MCP registry; update them there.

Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.