Skip to main content
You need to configure the tools that your agent uses in production in the config file for running evaluating your LLM and running simulations via CLI. For a primer on tools, refer to Tools. Each tool has:
KeyTypeDescription
type"structured_output" or "webhook"Either a structured output tool or a webhook tool
namestringUnique identifier for the tool (ideally, an intuitive name that matches its purpose)
descriptionstringDescription shown to the LLM (explains the purpose of the tool and when it should be called)
parametersarraySchema of the expected output (required when type is "structured_output")
webhookobjectConfiguration for webhook tools (required when type is "webhook")

​
Structured output tools

Structured output tools extract structured data from the conversation. The agent calls the tool and returns the data in the format defined by the parameters schema. 
{
  "type": "structured_output",
  "name": "log_interaction",
  "description": "Log customer interaction details",
  "parameters": [
    {
      "id": "interaction_type",
      "type": "string",
      "description": "Type of interaction: order_inquiry, return_request, complaint",
      "required": true
    },
    {
      "id": "sentiment",
      "type": "string",
      "description": "Customer sentiment: positive, neutral, or negative",
      "required": true
    }
  ]
}
Any valid JSON schema can be used as the parameters schema.

​
Webhook tools

Webhook tools make HTTP requests to external APIs. When the agent calls a webhook tool, Calibrate makes the HTTP request and returns the response to the agent. 
{
  "type": "webhook",
  "name": "check_order",
  "description": "Check the status of a customer order",
  "parameters": [],
  "webhook": {
    "method": "GET",
    "url": "https://api.example.com/orders",
    "timeout": 10,
    "headers": [{ "name": "Authorization", "value": "Bearer API_KEY" }],
    "queryParameters": [
      {
        "id": "order_id",
        "type": "string",
        "description": "The customer's order ID",
        "required": true
      }
    ],
    "body": {
      "description": "Request body",
      "parameters": [
        {
          "id": "data",
          "type": "string",
          "description": "Form data",
          "required": true
        }
      ]
    }
  }
}
The webhook object has:
KeyTypeDescription
methodstringHTTP method: GET, POST, PUT, PATCH, DELETE
urlstringThe endpoint URL
timeoutnumberTime in seconds to wait for the response
headersarray(Optional) headers to send with the request
queryParametersarray(Optional) query parameters (each follows the same schema as parameters)
bodyobject(Optional) request body (required for POST, PUT, PATCH methods)