Agents

Create agent

Create an agent to test inside Calibrate or connect your existing agent to Calibrate

calibrate agents create [flags]

Options

Option	Type	Description
`-c, --config-param`	type	Agent behavioral config. The keys depend on type.

Examples Agent within Calibrate

calibrate agents create \
  --name "Support Agent" \
  --type "agent" \
  --config-param '{"system_prompt":"You are a helpful assistant.","llm":{"model":"google/gemini-2.5-flash"},"stt":{"provider":"google"},"tts":{"provider":"google"},"settings":{"agent_speaks_first":false,"max_assistant_turns":10}}'

Connect OpenAI-compatible agent

calibrate agents create \
  --name "My Hosted Agent" \
  --type "connection" \
  --config-param '{"agent_url":"https://api.example.com/v1/chat/completions","agent_headers":{"Authorization":"Bearer <token>"},"benchmark_provider":"openrouter"}'

For more details, see Agent connections.

Get agent

Get one agent by its ID

calibrate agents get [flags]

Options

Option	Type	Description
`-a, --agent-uuid`	string	Required. The agent to retrieve

Examples

calibrate agents get --agent-uuid f47ac10b-58cc-4372-a567-0e02b2c3d479

Link evaluators to agent

Link one or more existing evaluators to an agent, skipping any already linked

calibrate agents link-evaluators [flags]

Options

Option	Type	Description
`-a, --agent-uuid`	string	Required. The agent to link the evaluators to
`-e, --evaluator-ids`	stringArray	Required. The evaluators to link to the agent. Ones that are already linked are skipped. Each must be one you created or a built-in default

Examples

calibrate agents link-evaluators --agent-uuid f47ac10b-58cc-4372-a567-0e02b2c3d479 --evaluator-ids '["f47ac10b-58cc-4372-a567-0e02b2c3d479"]'

List agents

Get the list of all your agents

calibrate agents list [flags]

Options

Option	Type	Description
`-l, --limit`	string	Maximum number of items to return. Omit for no limit (all items)
`--offset`	int	Number of items to skip before returning results
`--q`	name	Case-insensitive substring search on name. Blank is a no-op

Examples

calibrate agents list

List agent evaluators

List evaluators linked to an agent

calibrate agents list-evaluators [flags]

Options

Option	Type	Description
`-a, --agent-uuid`	string	Required. The agent whose evaluators to list
`-l, --limit`	string	Maximum number of items to return. Omit for no limit (all items)
`--offset`	int	Number of items to skip before returning results
`--q`	name	Case-insensitive substring search on name. Blank is a no-op

Examples

calibrate agents list-evaluators --agent-uuid f47ac10b-58cc-4372-a567-0e02b2c3d479

Resolve agent names to IDs

Get the IDs for your agents by their names

calibrate agents resolve [flags]

Options

Option	Type	Description
`-n, --names`	stringArray	Required. Agent names to resolve to IDs

Examples

calibrate agents resolve --names '["my-agent","support-bot"]'

Update agent

Update an agent’s configuration

calibrate agents update [flags]

Options

Option	Type	Description
`-a, --agent-uuid`	string	Required. The agent to update
`-c, --config-param`	type	Agent behavioral config. The keys depend on type.

Examples

calibrate agents update --agent-uuid f47ac10b-58cc-4372-a567-0e02b2c3d479

Verify agent connection

Verify an agent’s connection and persist the result when successful

calibrate agents verify-connection [flags]

Options

Option	Type	Description
`-a, --agent-uuid`	string	Required. The agent whose connection to verify
`--messages`	string	Sample chat messages to send during verification. Omit to use the default probe
`--model`	string	Model to verify. Omit for a basic connection check. Provide it for a model-specific check before benchmarking that model

Examples

calibrate agents verify-connection --agent-uuid f47ac10b-58cc-4372-a567-0e02b2c3d479

Getting started

Guides

Create agent

Get agent

Link evaluators to agent

List agents

List agent evaluators

Resolve agent names to IDs

Update agent

Verify agent connection

​Create agent

​Get agent

​Link evaluators to agent

​List agents

​List agent evaluators

​Resolve agent names to IDs

​Update agent

​Verify agent connection

Create agent

Get agent

Link evaluators to agent

List agents

List agent evaluators

Resolve agent names to IDs

Update agent

Verify agent connection