Overview - Calibrate

The Calibrate REST API gives you programmatic access to the full evaluation loop — create and connect agents, author test cases and evaluators, link tests to agents, launch test runs and benchmarks, poll results, and route runs to human reviewers with annotation tasks.

What you can build

Resource	What it does
Agents	Create, update, list, and verify-connect the agents under evaluation
Tests	Author test cases one at a time or in bulk, and link them to agents
Evaluators	Define and version the LLM judges that score runs
Agent tests	Launch runs and benchmarks, then poll status and per-test-case results
Annotation tasks	Route run outputs to human reviewers and measure inter-rater agreement

Getting started

Get your API key

Obtain your API key from the Calibrate dashboard. See the API keys guide for detailed setup instructions.

Authenticate your requests

Pass your key in the X-API-Key header on every request:

curl https://api.calibrate.artpark.ai/agents \
  -H "X-API-Key: your_api_key"

List agents

Enumerate every agent in your org:

curl https://api.calibrate.artpark.ai/agents \
  -H "X-API-Key: your_api_key"

Run tests

Launch a test run for an agent:

curl -X POST "https://api.calibrate.artpark.ai/agent-tests/agent/{agent_uuid}/run" \
  -H "X-API-Key: your_api_key" \
  -H "Content-Type: application/json" \
  -d '{}'

Since no test_uuids were passed, all linked tests for the agent will be run. The response returns a task_id:

{
  "task_id": "abc123",
  "status": "queued",
  "dataset_id": null,
  "dataset_name": null
}

Poll results

Poll the returned task_id for status and results:

curl "https://api.calibrate.artpark.ai/agent-tests/run/{task_id}" \
  -H "X-API-Key: your_api_key"

Base URL

https://api.calibrate.artpark.ai

OpenAPI specification

The public API schema is published at an unauthenticated endpoint.

# Fetch the live spec (JSON)
curl -s https://api.calibrate.artpark.ai/public-api/openapi.json

Authentication

All API requests require an X-API-Key header.

​What you can build

​Getting started

​Base URL

​OpenAPI specification

​Authentication

What you can build

Getting started

Base URL

OpenAPI specification

Authentication