Create test config

Create a Supabase-backed v1 test config and immutable judge membership for a dataset.

POST/v1/projects/{projectId}/test-configs

Authorization

AuthorizationPersonal API key · headerrequired

Your personal API key from Settings > Personal API Keys, sent as Bearer <key>.

X-Organization-IdOrganization ID · headerrequired

The organization to act in. Copy it with Copy organization ID in the organization menu or the settings sidebar.

Path parameters

projectIdstringrequired

Request body

requiredapplication/json

namestringrequired

descriptionstring | null

Show properties

Any of:

string

string

null

null

dataset_idstring

dataset_namestring

judge_idsstring[]

min items 1

judgesobject[]

min items 1

Show properties

Array of object

judge_idstring

namestring

Responses

200

Response for status 200

test_configobjectrequired

Show properties

idstringrequired

organization_idstring

project_idstringrequired

dataset_idstringrequired

namestringrequired

descriptionstring | null

Show properties

Any of:

string

string

null

null

created_atstring

updated_atstring

user_idstringrequired

judgesobject[]

Show properties

Array of object

idstring

organization_idstring

project_idstring

test_config_idstring

judge_idstringrequired

created_atstring

user_idstringrequired

judgeobject

Show properties

idstringrequired

namestringrequired

judge_typestringrequired

score_typestringrequired

409

Response for status 409

errorstringrequired

messagestring

codestring

detailsstring[]

422

Response for status 422

errorstringrequired

messagestring

codestring

detailsstring[]

Try it

Server

Base URLCustom base URLOverrides the selected server when set.

Authorization

Personal API keyOrganization IDRemember on this deviceInclude my values in samples

Parameters

projectIdpath*

Bodyapplication/json

Request body

Request

curl -X POST 'https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-configs' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'X-Organization-Id: YOUR_ORGANIZATION_ID' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "string",
  "description": "string",
  "dataset_id": "string",
  "dataset_name": "string",
  "judge_ids": [
    "string"
  ],
  "judges": [
    {
      "judge_id": "string",
      "name": "string"
    }
  ]
}'

const response = await fetch("https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-configs", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "X-Organization-Id": "YOUR_ORGANIZATION_ID",
    "Accept": "application/json",
    "Content-Type": "application/json"
  },
  body: "{\n  \"name\": \"string\",\n  \"description\": \"string\",\n  \"dataset_id\": \"string\",\n  \"dataset_name\": \"string\",\n  \"judge_ids\": [\n    \"string\"\n  ],\n  \"judges\": [\n    {\n      \"judge_id\": \"string\",\n      \"name\": \"string\"\n    }\n  ]\n}"
});

import requests

response = requests.post(
    "https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-configs",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "X-Organization-Id": "YOUR_ORGANIZATION_ID",
        "Accept": "application/json",
        "Content-Type": "application/json"
    },
    data="{\n  \"name\": \"string\",\n  \"description\": \"string\",\n  \"dataset_id\": \"string\",\n  \"dataset_name\": \"string\",\n  \"judge_ids\": [\n    \"string\"\n  ],\n  \"judges\": [\n    {\n      \"judge_id\": \"string\",\n      \"name\": \"string\"\n    }\n  ]\n}",
)

Response

{
  "test_config": {
    "id": "string",
    "organization_id": "string",
    "project_id": "string",
    "dataset_id": "string",
    "name": "string",
    "description": "string",
    "created_at": "string",
    "updated_at": "string",
    "user_id": "string",
    "judges": [
      {
        "id": "string",
        "organization_id": "string",
        "project_id": "string",
        "test_config_id": "string",
        "judge_id": "string",
        "created_at": "string",
        "user_id": "string",
        "judge": {
          "id": "string",
          "name": "string",
          "judge_type": "string",
          "score_type": "string"
        }
      }
    ]
  }
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}