LLM evals as MCP tools: score outputs for faithfulness, relevancy, and hallucination.
Connect Runs on your machine
claude mcp add retrieval-mcp -e ANTHROPIC_API_KEY=<ANTHROPIC_API_KEY> -e RETRIEVAL_JUDGE_BACKEND=<RETRIEVAL_JUDGE_BACKEND> -e RETRIEVAL_JUDGE_MODEL=<RETRIEVAL_JUDGE_MODEL> -e RETRIEVAL_BUDGET_USD=<RETRIEVAL_BUDGET_USD> -- docker run -i --rm ghcr.io/hcarrillo001/retrieval-mcp:1.0.0{
"mcpServers": {
"retrieval-mcp": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
],
"env": {
"ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>",
"RETRIEVAL_JUDGE_BACKEND": "<RETRIEVAL_JUDGE_BACKEND>",
"RETRIEVAL_JUDGE_MODEL": "<RETRIEVAL_JUDGE_MODEL>",
"RETRIEVAL_BUDGET_USD": "<RETRIEVAL_BUDGET_USD>"
}
}
}
}{
"mcpServers": {
"retrieval-mcp": {
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
],
"env": {
"ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>",
"RETRIEVAL_JUDGE_BACKEND": "<RETRIEVAL_JUDGE_BACKEND>",
"RETRIEVAL_JUDGE_MODEL": "<RETRIEVAL_JUDGE_MODEL>",
"RETRIEVAL_BUDGET_USD": "<RETRIEVAL_BUDGET_USD>"
}
}
}
}{
"servers": {
"retrieval-mcp": {
"type": "stdio",
"command": "docker",
"args": [
"run",
"-i",
"--rm",
"ghcr.io/hcarrillo001/retrieval-mcp:1.0.0"
],
"env": {
"ANTHROPIC_API_KEY": "<ANTHROPIC_API_KEY>",
"RETRIEVAL_JUDGE_BACKEND": "<RETRIEVAL_JUDGE_BACKEND>",
"RETRIEVAL_JUDGE_MODEL": "<RETRIEVAL_JUDGE_MODEL>",
"RETRIEVAL_BUDGET_USD": "<RETRIEVAL_BUDGET_USD>"
}
}
}
}Replace the values in angle brackets with your own. Keys stay on your machine; we never see them.
Tools
This server runs locally, so its tools are listed by your MCP client once it starts.
Environment variables
| Name | Description |
|---|---|
ANTHROPIC_API_KEYOptional, secret | Judge model API key. Not needed if you set RETRIEVAL_JUDGE_BACKEND=ollama to run the judge locally. |
RETRIEVAL_JUDGE_BACKENDOptional | Which judge to score with: anthropic (default), openai, or ollama for a fully local judge. |
RETRIEVAL_JUDGE_MODELOptional | Judge model id, e.g. claude-sonnet-4-6 or qwen2.5:14b for Ollama. |
RETRIEVAL_BUDGET_USDOptional | Spend cap for judge calls. Unset means no cap. |
Details
| Package | ghcr.io/hcarrillo001/retrieval-mcp:1.0.0 (OCI, stdio) |
| First published | 24 Aug 2026 |
| Last updated | 24 Aug 2026 |
| Registry name | io.github.hcarrillo001/retrieval-mcp |
More ai and media servers See all
Listing data from the official MCP registry. Status and tools from NestDaddy's daily check. Browse all MCP servers.