Skip to main content
The /onboard skill guides you through setting up your first Calibrate evaluation step by step. It’s an orchestrator — each phase delegates to a feature skill and produces a real Calibrate object, so you finish with a working evaluation, not a tutorial.

​
Quick start

1

Install the skills

See Agent skills for the full pack.
2

Open your AI coding agent

Claude Code, Cursor, Codex, Windsurf, or any Agent Skills–compatible agent.
3

Run the onboarding skill

The skill handles everything from there — including installing the CLI and authenticating if you haven’t already.
The onboarding is resumable and adaptive: progress is saved to .calibrate/onboard.md, so a later session picks up at the first incomplete phase, and the skill compresses or skips phases you’ve already answered.

​
What it creates

By the end you have a real, inspectable evaluation on Calibrate:

​
The workflow

/onboard walks these phases in order for a first-time user, and lets an expert compress or skip any they’ve already answered.
1

Setup

Confirms the CLI is installed and authenticated (calibrate whoami, else calibrate login) and scaffolds the resumable state file.
2

Connect the agent

Asks what the agent does, who uses it, and where it fails today, then delegates to /connect-agent to register and verify the connection.
3

Build the tests

Turns your goal into a falsifiable hypothesis, then delegates to /build-test-suite (or /import-dataset if you already have a dataset) to author and upload test cases.
4

Design the evaluators

Only if a case judges response quality — delegates to /design-evaluator to create a judge. Skipped entirely when every case is a deterministic tool_call.
5

Run

Delegates to /run-tests to link the tests, run them, poll for results, and present failures first.
6

Calibrate the judge

Offered, not forced — when you care whether the scores are trustworthy, /calibrate-evaluator measures human ↔ judge agreement and tunes the judge until it matches humans. This is the step that makes it Calibrate, not just a test runner.
7

Scale and wire into CI

Delegates to /benchmark-models to compare models, and — when you mention PRs, releases, or shipping — proposes wiring calibrate agent-tests run into CI in agent mode.

​
Evaluation is a loop

After the first run, /onboard expects one of: expand the test set when a defect surfaces, tune the hypothesis when the bar is too loose or strict, add a model, or tune the judge when agreement is too low. Each pass appends to the state file so prior baselines stay inspectable — stop when you’ve shipped or wired it into CI.

​
Next steps

Browse the skills

See every skill /onboard orchestrates, and invoke them on their own.

Connect the MCP server

Give your agent native Calibrate tools alongside the skills.