Tests

Bulk create tests

Create many test cases at once and link them to your agents

calibrate tests bulk-create [flags]

Options

Option	Type	Description
`-a, --agent-uuids`	string	IDs of agents to link every created test to. Omit to link none
`-l, --language`	config.settings.language	Language written to each test’s config.settings.language. Omit to leave unset
`--tests`	string	Required. Test items to create, at most 500 per request, with names unique within the batch
`--type`	response	What the test judges:

Examples

calibrate tests bulk-create --type tool_call --tests '[{"name":"<value>","conversation_history":[{"role":"tool"}],"evaluators":[{"evaluator_uuid":"f47ac10b-58cc-4372-a567-0e02b2c3d479","variable_values":{"criteria":"The reply must cite the refund window"} }]}]'

Create test

Create a test that runs your agent against a conversation and evaluates its answer quality or the tools it calls

calibrate tests create [flags]

Options

Option	Type	Description
`-c, --config-param`	history	The calibrate test config. Three top-level keys.

Examples

calibrate tests create --name <value> --type tool_call

Get test

Get an agent test case by its ID

calibrate tests get [flags]

Options

Option	Type	Description
`-t, --test-uuid`	string	Required. Test to retrieve

Examples

calibrate tests get --test-uuid b1c2d3e4-f5a6-7890-bcde-f12345678901

List tests

List all the test cases for your agents

calibrate tests list [flags]

Options

Option	Type	Description
`-l, --limit`	string	Maximum number of items to return. Omit for no limit (all items)
`--offset`	int	Number of items to skip before returning results
`--q`	name	Case-insensitive substring search on name. Blank is a no-op

Examples

calibrate tests list

Update test

Update an agent test case

calibrate tests update [flags]

Options

Option	Type	Description
`-c, --config-param`	history	The calibrate test config. Three top-level keys.

Examples

calibrate tests update --test-uuid b1c2d3e4-f5a6-7890-bcde-f12345678901

Getting started

Guides

Bulk create tests

Create test

Get test

List tests

Update test

​Bulk create tests

​Create test

​Get test

​List tests

​Update test

Bulk create tests

Create test

Get test

List tests

Update test