> ## Documentation Index
> Fetch the complete documentation index at: https://penseapp.vercel.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Get started with Calibrate's Python SDK

The Calibrate Python SDK is a typed client for the full evaluation loop — manage agents, tests, and evaluators, launch runs and benchmarks, and set up human-in-the-loop annotation tasks, all from Python.

## Install

```bash theme={null}
uv add calibrate-sdk
```

## Auth

Create an API key under [**Workspace settings → API keys**](https://calibrate.artpark.ai/workspace-settings?tab=api-keys) — see the [API keys guide](/docs/reference/api-keys).

```python theme={null}
from calibrate import Calibrate

client = Calibrate(api_key="your_api_key")

for agent in client.agents.list():
    print(agent.uuid, agent.name)
```

## What you can do

Every REST resource maps to a namespace on the client:

| Namespace                 | Methods                                                                                                                                             |
| ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `client.agents`           | `create`, `list`, `get`, `resolve`, `update`, `verify_connection`                                                                                   |
| `client.tests`            | `create`, `bulk_create`, `list`, `get`, `update`                                                                                                    |
| `client.evaluators`       | `create`, `create_version`, `list`, `get`                                                                                                           |
| `client.agent_tests`      | `link`, `run`, `run_batch`, `benchmark`, `list_for_agent`, `list_runs_for_agent`, `get_run`, `get_benchmark`                                        |
| `client.annotation_tasks` | `create`, `add_items`, `update_items`, `link_evaluator`, `create_evaluator_run`, `get_evaluator_run`, `get_agreement`, `get_summary`, `list`, `get` |

## Resources

<CardGroup cols={1}>
  <Card title="Source" icon="github" href="https://github.com/dalmia/calibrate-python-sdk">
    calibrate-python-sdk
  </Card>
</CardGroup>