Endpoints

Configure and manage API endpoints that your tests execute against.

What are Endpoints? Endpoints represent the AI services or APIs that you want to test. They define how Rhesis connects to your application, sends test inputs, and receives responses for evaluation.

Why Endpoints?

Endpoints enable you to test AI applications without hardcoding API details into every test. They provide:

Reusability: Configure once, use across hundreds of tests
Flexibility: Switch between models, environments, or providers without changing tests
Comparison: Run identical tests against different endpoints to compare performance
Version Control: Track configuration changes and their impact on test results
Security: Centralize API keys and credentials in one place

Understanding Endpoints

An endpoint in Rhesis is a complete configuration for calling an external API. When you run tests, Rhesis:

Takes your test prompt or input
Formats it according to your endpoint’s request template
Sends the request to your API
Receives the response
Evaluates the response against your metrics

Think of endpoints as the bridge between your tests and the AI system you’re evaluating.

Platform-Managed Variables

Rhesis uses a set of platform-managed variables to standardize communication between your tests and your API. You map these variables to the fields your endpoint expects (in request templates) and the fields your endpoint returns (in response mappings). This abstraction lets Rhesis work with any API regardless of its specific field names.

Request Variables

These variables are available in your request body templates using Jinja2 syntax (\{\{ variable \}\}). Rhesis populates them automatically when running tests.

Variable	Required	Description
`input`	Yes	The user query or test prompt. This is the primary text sent to your AI endpoint.
`conversation_id`	No	Conversation tracking identifier for multi-turn endpoints (both stateful and stateless). Rhesis uses this to track conversations across turns.
`messages`	No	Full conversation history as an array of message objects. Used for stateless endpoints that require the entire history with every request.
`system_prompt`	No	System prompt text. Rhesis prepends it to the `messages` array and strips it from the final request body before sending.

Response Variables

These variables are extracted from your API response using JSONPath or Jinja2 expressions in the response mapping. Rhesis uses them for evaluation, display, and conversation tracking.

Variable	Required	Description
`output`	Yes	The main response text from your API. This is what Rhesis evaluates against your metrics.
`context`	No	Additional context or reasoning provided by the response (e.g., retrieved documents, sources).
`metadata`	No	Arbitrary metadata about the response (e.g., model version, token counts). Stored but not actively evaluated.
`tool_calls`	No	Tool or function call results returned by the API.
Conversation ID fields	For conversational	Any of the recognized conversation identifiers: `conversation_id`, `session_id`, `thread_id`, `chat_id`, `dialog_id`, `dialogue_id`, `context_id`, `interaction_id`. Map one of these for both stateful and stateless multi-turn endpoints so Rhesis can track conversations across turns.

You can include additional custom fields in both request templates and response mappings. Custom fields are passed through and stored, but Rhesis does not actively use them for evaluation or conversation management.

How Mapping Works

Request and response mappings tell Rhesis how to translate between its standard variables and your API’s specific field names.

Request mapping uses Jinja2 templates to place platform variables into your API’s expected request body structure:

request-mapping-example.json
{
  "model": "gpt-4",
  "messages": [
    {
      "role": "user",
      "content": "{{ input }}"
    }
  ],
  "temperature": 0.7
}

Response mapping uses JSONPath expressions to extract values from your API’s response into platform variables:

response-mapping-example.json
{
  "output": "$.choices[0].message.content",
  "metadata": "$.usage"
}

For full details on template syntax, filters, and advanced mapping techniques, see Single-Turn Endpoints.

Creating an Endpoint

New to mapping? Use the Auto-Configure feature to let AI generate your request and response mappings automatically. Just paste a curl command, code snippet, or API documentation and Rhesis handles the rest.

Manual Configuration

Create an endpoint from scratch with full control over all settings.

Configure the endpoint name, description, project assignment, and environment. Then set up the request by providing the API URL, protocol (REST or WebSocket), and HTTP method.

Request Headers

Define authentication and other required headers in JSON format:

headers.json
{
  "Authorization": "Bearer {{ auth_token }}",
  "Content-Type": "application/json"
}

The auth_token variable is a platform-managed placeholder that Rhesis automatically replaces with the authentication token you configure in the endpoint settings UI. You can reference it using Jinja2 syntax (\{\{ auth_token \}\}) or the legacy \{API_KEY\} / \{auth_token\} placeholders with single braces.

If you configure an authentication token in the UI but do not include an Authorization header explicitly, Rhesis automatically adds Authorization: Bearer <token> to the request headers.

Request and Response Mappings

Configure your request body template and response mappings to bridge platform variables with your API’s fields. Rhesis supports two endpoint patterns depending on how your API handles conversations:

Single-Turn Endpoints: Standard request/response mapping with Jinja2 templates and JSONPath. Covers most use cases including simple query-response APIs.
Multi-Turn Conversations: Conversation tracking for stateful endpoints (your API manages session state) and message history for stateless endpoints (Rhesis manages conversation state).
Mapping Examples: Practical examples for OpenAI, Anthropic, Google Gemini, RAG pipelines, WebSocket endpoints, and more.

SDK Endpoints

If your AI logic lives in Python functions, you can register them as endpoints directly from code using the Rhesis SDK. Decorate your functions with @endpoint and the SDK handles registration and mapping automatically. See SDK Endpoints for details, or the full SDK Connector reference for advanced patterns.

Testing Your Endpoint

Before running full test suites, verify your endpoint configuration works correctly. Navigate to the Test Connection tab, enter sample input data, and click Test Endpoint. Review the response to ensure it returns expected data.

Test Connection

Managing Endpoints

Viewing Endpoints

The Endpoints page displays all your configured endpoints organized by project. Each entry shows the project icon and name, the endpoint name, the protocol (REST or WebSocket), and the environment (development, staging, or production). Click any endpoint to view its full configuration and execution history.

Editing Endpoints

Open the endpoint details page, click Edit, modify any configuration fields, and click Save. Changes take effect immediately for new test runs.

Duplicating Endpoints

To create a copy of an existing endpoint, open the endpoint details page and click Duplicate. Rhesis creates a new endpoint with the same configuration and appends “(Copy)” to the name. You can also select multiple endpoints from the grid and duplicate them in bulk.

Deleting Endpoints

Select one or more endpoints from the grid and click Delete.

Important: Deleting an endpoint does not delete associated test configurations or historical test results. Your test data remains intact, but you cannot execute new tests with a deleted endpoint.

Using Endpoints in Tests

Executing Test Sets

When you run a test set, select which endpoint to execute it against, configure the execution mode (parallel or sequential), and click Run Tests. Rhesis sends each test’s prompt through the endpoint and evaluates responses against your configured metrics.

Multiple Endpoints

Creating multiple endpoints opens up powerful testing scenarios. You can run the same tests against different AI models to compare their performance, set up separate endpoints for each environment to validate changes before production deployment, configure A/B tests to compare response quality across different settings, or use different endpoints for load and performance testing. Each test run is independent, allowing you to analyze differences in behavior, quality, and performance across your various configurations.

Environment Management

Organize endpoints by environment to match your deployment workflow.

Development endpoints typically point to local or development servers where you can iterate quickly, debug issues, and test configuration changes without any risk to production systems.

Staging endpoints connect to pre-production systems for validation, integration testing, and performance verification before promoting changes to production.

Production endpoints represent live production APIs. Use these for regression testing and quality monitoring of your deployed AI systems. These endpoints require extra care when modifying.

Environment tags help you quickly identify which endpoints are production-critical and which are safe for experimentation.

Next Steps

Try Auto-Configure to let AI generate mappings from a curl command or code snippet
Learn about Single-Turn Endpoints for request/response mapping details
Configure Multi-Turn Conversations for conversational AI testing
Register Python functions with SDK Endpoints
Browse Mapping Examples for real-world configuration patterns
Generate Tests with AI to create comprehensive coverage
Define Metrics to evaluate response quality