> ## Documentation Index
> Fetch the complete documentation index at: https://penseapp.vercel.app/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Introduction

> Learn about Calibrate and how to get started.

[Calibrate](https://calibrate.artpark.ai) is an open-source evaluation platform for AI agents — text or voice.

<iframe className="w-full aspect-video rounded-xl" src="https://www.youtube.com/embed/_VS8KQbBxKs" title="Calibrate Overview" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowFullScreen />

Testing AI agents manually is slow, inconsistent, and doesn't scale. Every agent has to get several things right, turn after turn:

* Reasoning about the user's intent against the context and purpose of the conversation
* Calling the right tools with the right arguments in the right order
* Generating the right response
* Handling multiple languages and code-switching

It can be hard to even define what "the right response" means.

If you're building a **voice agent**, there are further challenges on top:

* Transcribing the user's speech accurately and efficiently
* Producing natural-sounding speech back to the user
* Detecting when the user has finished speaking or has only paused
* Handling interruptions while the agent is speaking
* Keeping the turnaround time low
* Handling varying accents and dialects

and a lot more.

There is no simple way to evaluate each of these across every provider on your own dataset. Even when individual pieces work well, it is hard to tell whether the agent as a whole will behave in production. This leads to many teams deploying with a lot of uncertainty and risk, and their users having to deal with poor experiences.

Calibrate solves this by letting you evaluate the whole agent, whichever pieces it uses:

* `Text agents (LLMs)`: Evaluate the response quality and tool calling of your LLMs for multi-turn conversations, define custom evaluators, and find the best LLM for your agent
* `Voice agents (STT & TTS)`: Benchmark speech-to-text and text-to-speech providers (Google, Sarvam, ElevenLabs and more) on your dataset across 10+ indic languages, using metrics optimised for agentic use cases
* `Simulations`: Simulate realistic conversations using user personas and scenarios to surface failure modes — including interruptions — before you deploy
* `Human alignment`: Collect human labels and iteratively align your LLM judges to them, so you can automate monitoring reliably

Calibrate helps you continuously improve your agent, ensure a bug never repeats itself, and deploy with confidence.

## Get started

<CardGroup cols={2}>
  <Card title="LLM tests" icon="brain" href="/docs/quickstart/text-to-text">
    Create test suites that verify model responses and tool calling behavior
  </Card>

  <Card title="Simulations" icon="comments" href="/docs/quickstart/simulations">
    Simulate agent conversations with customizable personas and scenarios
  </Card>

  <Card title="Speech to Text" icon="microphone" href="/docs/quickstart/speech-to-text">
    Compare transcription accuracy across multiple providers on your dataset
  </Card>

  <Card title="Text to Speech" icon="volume-high" href="/docs/quickstart/text-to-speech">
    Benchmark generated audio quality across multiple providers
  </Card>
</CardGroup>

## Learn more

<CardGroup cols={2}>
  <Card title="Core Concepts" icon="book" href="/docs/core-concepts/speech-to-text">
    Understand how Calibrate helps you evaluate effectively
  </Card>

  <Card title="Agents" icon="robot" href="/docs/agents/overview">
    Give your AI coding agent the skills to evaluate on Calibrate
  </Card>

  <Card title="CLI" icon="terminal" href="/docs/cli/calibrate/overview">
    Drive the full evaluation loop from your terminal
  </Card>

  <Card title="Python SDK" icon="python" href="/docs/sdk/overview">
    Script agents, tests, runs, and benchmarks from Python
  </Card>

  <Card title="MCP" icon="server" href="/docs/mcp/overview">
    Give your coding agent native Calibrate tools over MCP
  </Card>

  <Card title="Integrations" icon="plug" href="/docs/integrations/stt">
    See all supported providers for different components
  </Card>
</CardGroup>