Score a test run

Triggers scoring of an existing run’s traces with the test suite’s latest published version, which may be the run’s own, without re-running the agent. Returns a new run with the cases that have an output and an unchanged input, which links back through scored_from_run_id. Its cases are scored in the background; poll the run for progress.

POST/v1/projects/{projectId}/test-suites/runs/{runId}/score

Authorization

AuthorizationPersonal API key · headerrequired

Your personal API key from Settings > Personal API Keys, sent as Bearer <key>.

X-Organization-IdOrganization ID · headerrequired

The organization to act in. Copy it with Copy organization ID in the organization menu or the settings sidebar.

Path parameters

projectIdstring<uuid>required

runIdstring<uuid>required

Responses

200

Response for status 200

run_idstring<uuid>required

The new run.

404

Response for status 404

errorstringrequired

messagestring

codestring

detailsstring[]

409

Response for status 409

errorstringrequired

messagestring

codestring

detailsstring[]

422

Response for status 422

errorstringrequired

messagestring

codestring

detailsstring[]

Try it

Server

Base URLCustom base URLOverrides the selected server when set.

Authorization

Personal API keyOrganization IDRemember on this deviceInclude my values in samples

Parameters

projectIdpath*runIdpath*

Request

curl -X POST 'https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-suites/runs/YOUR_RUN_ID/score' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'X-Organization-Id: YOUR_ORGANIZATION_ID' \
  -H 'Accept: application/json'

const response = await fetch("https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-suites/runs/YOUR_RUN_ID/score", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "X-Organization-Id": "YOUR_ORGANIZATION_ID",
    "Accept": "application/json"
  }
});

import requests

response = requests.post(
    "https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/test-suites/runs/YOUR_RUN_ID/score",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "X-Organization-Id": "YOUR_ORGANIZATION_ID",
        "Accept": "application/json"
    },
)

Response

{
  "run_id": "dded282c-8ebd-44cf-8ba5-9a234973d1ec"
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}