List judges

List judges and their selected configuration. Defaults to the latest configuration; use version=‘production’ for production judges and enabled judge labels.

GET/v1/projects/{projectId}/judges

Authorization

AuthorizationPersonal API key · headerrequired

Your personal API key from Settings > Personal API Keys, sent as Bearer <key>.

X-Organization-IdOrganization ID · headerrequired

The organization to act in. Copy it with Copy organization ID in the organization menu or the settings sidebar.

Path parameters

projectIdstringrequired

Query parameters

versionstring

Version to list. Defaults to 'latest'; use 'production' for enabled judge labels.

Allowed:latestproduction

summaryboolean

Return only IDs, names, types, the selected version, and online-evaluation settings. Defaults to false.

Responses

200

Response for status 200

judgesobject[]required

Show properties

Any of:

object[]

Array of object

idstringrequired

namestringrequired

judge_typestringrequired

Allowed:promptcustomagentdeterministic

score_typestringrequired

Allowed:numericbinarycategorical

judge_major_versionnumberrequired

judge_minor_versionnumberrequired

is_productionbooleanrequired

online_evaluation_modestringrequired

Allowed:continuouson_demand

online_environment_namesstring[] | nullrequired

Environments it scores. An empty list means all environments; null means none set.

Show properties

Any of:

string[]

Array of string

string

null

null

online_sampling_ratenumberrequired

object[]

Array of object

idstringrequired

namestringrequired

judge_descriptionstring | nullrequired

judge_typestringrequired

Agent judge (wire value prompt); Code judge (wire value custom); legacy Agent judge (wire value agent); Condition judge (wire value deterministic)

Allowed:promptcustomagentdeterministic

knowledge_file_idstring | nullrequired

score_typestringrequired

numeric, binary, or categorical. For binary judges, true means the judge applied its label, which the app shows under the judge's name. false means no label was applied. For classification judges, the value is the label, meaning the chosen option.

Allowed:numericbinarycategorical

updated_atstringrequired

judge_version_idstringrequired

judge_major_versionnumberrequired

judge_minor_versionnumberrequired

is_latestbooleanrequired

is_productionbooleanrequired

optimistic_eval_enabledbooleanrequired

reasoning_levelstring | nullrequired

Show properties

Any of:

string

string

null

null

outputsobject[]required

Labels this judge can apply. For binary judges, true means the judge applied its label, which the app shows under the judge's name. false means no label was applied. For classification judges, the value is the label, meaning the chosen option.

Show properties

Array of object

behavior_idstringrequired

namestringrequired

descriptionstringrequired

online_environment_namesstring[] | nullrequired

Environments it scores. An empty list means all environments; null means none set.

Show properties

Any of:

string[]

Array of string

string

null

null

online_evaluation_modestringrequired

Allowed:continuouson_demand

action_configobject | null

Show properties

Any of:

object

action_groupsobject[]required

Show properties

Array of object

idstringrequired

min length 1

namestringrequired

Action group display name; an empty string uses Action group 1, 2, etc.

enabledbooleanrequired

environment_namesstring[]required

Empty runs the group in every environment

whenobjectrequired

Judge result that runs the group. Categorical groups run when the result is any listed output name.

Show properties

One of:

object

kindstringrequired

object

kindstringrequired

valuebooleanrequired

object

kindstringrequired

valuesstring[]required

min items 1

object

kindstringrequired

operatorstringrequired

Allowed:ltlteeqgtegt

valuenumberrequired

frequencyobjectrequired

Show properties

countintegerrequired

min 1 · max 9007199254740991

window_secondsintegerrequired

min 1 · max 9007199254740991

cooldown_secondsintegerrequired

min 0 · max 9007199254740991

actionsobject[]required

Show properties

Array of object

One of:

object

idstringrequired

min length 1

enabledbooleanrequired

typestringrequired

configobjectrequired

Show properties

urlstringrequired

min length 1

secretstringrequired

min length 1

object

idstringrequired

min length 1

enabledbooleanrequired

typestringrequired

configobjectrequired

Show properties

email_addressstringrequired

min length 1

object

idstringrequired

min length 1

enabledbooleanrequired

typestringrequired

configobjectrequired

Show properties

team_idstring

min length 1

channel_idstring

min length 1

messagestring

custom_attribute_keysstring[]

object

idstringrequired

min length 1

enabledbooleanrequired

typestringrequired

configobjectrequired

Show properties

dataset_idstringrequired

min length 1

object

idstringrequired

min length 1

enabledbooleanrequired

typestringrequired

configobjectrequired

Show properties

judge_idsstring[]required

min items 1

null

null

online_sampling_ratenumberrequired

min 0 · max 100

online_span_triggersany | objectrequired

Show properties

Any of:

any

any

object

kindstringrequired

Allowed:andor

conditionsany[]required

online_session_scoringbooleanrequired

Try it

Server

Base URLCustom base URLOverrides the selected server when set.

Authorization

Personal API keyOrganization IDRemember on this deviceInclude my values in samples

Parameters

projectIdpath*versionqueryVersion to list. Defaults to 'latest'; use 'production' for enabled judge labels.summaryqueryReturn only IDs, names, types, the selected version, and online-evaluation settings. Defaults to false.

Request

curl -X GET 'https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/judges' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'X-Organization-Id: YOUR_ORGANIZATION_ID' \
  -H 'Accept: application/json'

const response = await fetch("https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/judges", {
  method: "GET",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "X-Organization-Id": "YOUR_ORGANIZATION_ID",
    "Accept": "application/json"
  }
});

import requests

response = requests.get(
    "https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/judges",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "X-Organization-Id": "YOUR_ORGANIZATION_ID",
        "Accept": "application/json"
    },
)

Response

{
  "judges": [
    {
      "id": "string",
      "name": "string",
      "judge_type": "prompt",
      "score_type": "numeric",
      "judge_major_version": 0,
      "judge_minor_version": 0,
      "is_production": true,
      "online_evaluation_mode": "continuous",
      "online_environment_names": [
        "string"
      ],
      "online_sampling_rate": 0
    }
  ]
}