Log evaluation results

Log evaluation results from SDK

POST/v1/projects/{projectId}/eval-results

Authorization

AuthorizationPersonal API key · headerrequired

Your personal API key from Settings > Personal API Keys, sent as Bearer <key>.

X-Organization-IdOrganization ID · headerrequired

The organization to act in. Copy it with Copy organization ID in the organization menu or the settings sidebar.

Path parameters

projectIdstringrequired

Request body

requiredapplication/json

resultsScoringResult[]required

Show properties

Array of ScoringResult

Any of:

ExampleScoringResult

scorers_dataobject[]required

Show properties

Array of object

scorer_namestringrequired

namestring | null

Show properties

Any of:

string

string

null

null

data_objectExamplerequired

trace_idstring | null

Show properties

Any of:

string

string

null

null

run_durationnumber | null

Show properties

Any of:

number

number

null

null

evaluation_costnumber | null

Show properties

Any of:

number

number

null

null

TraceScoringResult

scorers_dataobject[]required

Show properties

Array of object

scorer_namestringrequired

namestring

data_objectTraceSpanrequired

Show properties

organization_idstringrequired

project_idstringrequired

timestampstringrequired

trace_idstringrequired

span_idstringrequired

parent_span_idstring | nullrequired

Show properties

Any of:

string

string

null

null

trace_statestring | nullrequired

Show properties

Any of:

string

string

null

null

span_namestring | nullrequired

Show properties

Any of:

string

string

null

null

span_kindstring | nullrequired

Show properties

Any of:

string

string

null

null

service_namestring | nullrequired

Show properties

Any of:

string

string

null

null

resource_attributesobjectrequired

span_attributesobjectrequired

durationstringrequired

status_codenumberrequired

status_messagestring | nullrequired

Show properties

Any of:

string

string

null

null

eventsobject[]required

Show properties

Array of object

Timestampstringrequired

Namestringrequired

Attributesobjectrequired

linksstring | nullrequired

Show properties

Any of:

string

string

null

null

trace_idstring

run_durationnumber

evaluation_costnumber

runExampleEvaluationRunrequired

Show properties

idstring | null

Show properties

Any of:

string

string

null

null

project_idstringrequired

eval_namestringrequired

modelstring | null

Show properties

Any of:

string

string

null

null

created_atstring | null

Show properties

Any of:

string

string

null

null

user_idstring

scorersany[] | null

Show properties

Any of:

any[]

Array of any

any

null

null

custom_scorersBaseScorer[]required

default: []

Show properties

Array of BaseScorer

score_typestringrequired

namestringrequired

class_namestring | null

Show properties

Any of:

string

string

null

null

scorenumber | null

Show properties

Any of:

number

number

null

null

minimum_score_rangenumber

default: 0

maximum_score_rangenumber

default: 1

score_breakdownobject | null

Show properties

Any of:

object

object

null

null

reasonstring | object

Show properties

Any of:

string

string

object

object

successboolean | null

Show properties

Any of:

boolean

boolean

null

null

modelstring | null

Show properties

Any of:

string

string

null

null

errorstring | null

Show properties

Any of:

string

string

null

null

additional_metadataobject | null

Show properties

Any of:

object

object

null

null

userstring | null

Show properties

Any of:

string

string

null

null

server_hostedboolean | null

Show properties

Any of:

boolean

boolean

null

null

using_native_modelboolean | null

Show properties

Any of:

boolean

boolean

null

null

required_paramsstring[] | null

Show properties

Any of:

string[]

Array of string

string

null

null

strict_modeboolean | null

Show properties

Any of:

boolean

boolean

null

null

judgment_scorersScorerConfig[]required

default: []

Show properties

Array of ScorerConfig

score_typestring | null

Show properties

Any of:

string

string

null

null

namestringrequired

modelstring | null

Show properties

Any of:

string

string

null

null

required_paramsstring[] | null

Show properties

Any of:

string[]

Array of string

string

null

null

kwargsobject | null

Show properties

Any of:

object

object

null

null

result_typestring

Allowed:numericbinarycategorical

examplesExample[]required

min items 1

Show properties

Array of Example

example_idstringrequired

created_atstringrequired

namestring | null

Show properties

Any of:

string

string

null

null

conversation_idstring | null

Show properties

Any of:

string

string

null

null

trace_idstring | null

Show properties

Any of:

string

string

null

null

span_idstring | null

Show properties

Any of:

string

string

null

null

offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

session_idstring | null

Show properties

Any of:

string

string

null

null

session_trace_idsstring[]

agent_offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

trace_span_idstring | null

Show properties

Any of:

string

string

null

null

trace_idstring | null

Show properties

Any of:

string

string

null

null

Responses

200

Response for status 200

ui_results_urlstringrequired

422

Response for status 422

errorstringrequired

messagestring

codestring

detailsstring[]

Try it

Server

Base URLCustom base URLOverrides the selected server when set.

Authorization

Personal API keyOrganization IDRemember on this deviceInclude my values in samples

Parameters

projectIdpath*

Bodyapplication/json

Request body

{
  "results": [
    {
      "scorers_data": [
        {
          "scorer_name": "string",
          "reason": {
            "text": "string",
            "citations": [
              {
                "span_id": "string",
                "span_attribute": "string"
              }
            ]
          },
          "metadata": {
            "evaluation_model": "string",
            "evaluation_reasoning_level": "lite"
          },
          "error": "string",
          "score_type": "numeric",
          "num_value": 0,
          "minimum_score_range": 0,
          "maximum_score_range": 1
        }
      ],
      "name": "string",
      "data_object": {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      },
      "trace_id": "string",
      "run_duration": 0,
      "evaluation_cost": 0
    }
  ],
  "run": {
    "id": "string",
    "project_id": "string",
    "eval_name": "string",
    "model": "string",
    "created_at": "string",
    "user_id": "string",
    "scorers": [
      null
    ],
    "custom_scorers": [],
    "judgment_scorers": [],
    "examples": [
      {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    ],
    "trace_span_id": "string",
    "trace_id": "string"
  }
}

Request

curl -X POST 'https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'X-Organization-Id: YOUR_ORGANIZATION_ID' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "results": [
    {
      "scorers_data": [
        {
          "scorer_name": "string",
          "reason": {
            "text": "string",
            "citations": [
              {
                "span_id": "string",
                "span_attribute": "string"
              }
            ]
          },
          "metadata": {
            "evaluation_model": "string",
            "evaluation_reasoning_level": "lite"
          },
          "error": "string",
          "score_type": "numeric",
          "num_value": 0,
          "minimum_score_range": 0,
          "maximum_score_range": 1
        }
      ],
      "name": "string",
      "data_object": {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      },
      "trace_id": "string",
      "run_duration": 0,
      "evaluation_cost": 0
    }
  ],
  "run": {
    "id": "string",
    "project_id": "string",
    "eval_name": "string",
    "model": "string",
    "created_at": "string",
    "user_id": "string",
    "scorers": [
      null
    ],
    "custom_scorers": [],
    "judgment_scorers": [],
    "examples": [
      {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    ],
    "trace_span_id": "string",
    "trace_id": "string"
  }
}'

const response = await fetch("https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "X-Organization-Id": "YOUR_ORGANIZATION_ID",
    "Accept": "application/json",
    "Content-Type": "application/json"
  },
  body: "{\n  \"results\": [\n    {\n      \"scorers_data\": [\n        {\n          \"scorer_name\": \"string\",\n          \"reason\": {\n            \"text\": \"string\",\n            \"citations\": [\n              {\n                \"span_id\": \"string\",\n                \"span_attribute\": \"string\"\n              }\n            ]\n          },\n          \"metadata\": {\n            \"evaluation_model\": \"string\",\n            \"evaluation_reasoning_level\": \"lite\"\n          },\n          \"error\": \"string\",\n          \"score_type\": \"numeric\",\n          \"num_value\": 0,\n          \"minimum_score_range\": 0,\n          \"maximum_score_range\": 1\n        }\n      ],\n      \"name\": \"string\",\n      \"data_object\": {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      },\n      \"trace_id\": \"string\",\n      \"run_duration\": 0,\n      \"evaluation_cost\": 0\n    }\n  ],\n  \"run\": {\n    \"id\": \"string\",\n    \"project_id\": \"string\",\n    \"eval_name\": \"string\",\n    \"model\": \"string\",\n    \"created_at\": \"string\",\n    \"user_id\": \"string\",\n    \"scorers\": [\n      null\n    ],\n    \"custom_scorers\": [],\n    \"judgment_scorers\": [],\n    \"examples\": [\n      {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    ],\n    \"trace_span_id\": \"string\",\n    \"trace_id\": \"string\"\n  }\n}"
});

import requests

response = requests.post(
    "https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "X-Organization-Id": "YOUR_ORGANIZATION_ID",
        "Accept": "application/json",
        "Content-Type": "application/json"
    },
    data="{\n  \"results\": [\n    {\n      \"scorers_data\": [\n        {\n          \"scorer_name\": \"string\",\n          \"reason\": {\n            \"text\": \"string\",\n            \"citations\": [\n              {\n                \"span_id\": \"string\",\n                \"span_attribute\": \"string\"\n              }\n            ]\n          },\n          \"metadata\": {\n            \"evaluation_model\": \"string\",\n            \"evaluation_reasoning_level\": \"lite\"\n          },\n          \"error\": \"string\",\n          \"score_type\": \"numeric\",\n          \"num_value\": 0,\n          \"minimum_score_range\": 0,\n          \"maximum_score_range\": 1\n        }\n      ],\n      \"name\": \"string\",\n      \"data_object\": {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      },\n      \"trace_id\": \"string\",\n      \"run_duration\": 0,\n      \"evaluation_cost\": 0\n    }\n  ],\n  \"run\": {\n    \"id\": \"string\",\n    \"project_id\": \"string\",\n    \"eval_name\": \"string\",\n    \"model\": \"string\",\n    \"created_at\": \"string\",\n    \"user_id\": \"string\",\n    \"scorers\": [\n      null\n    ],\n    \"custom_scorers\": [],\n    \"judgment_scorers\": [],\n    \"examples\": [\n      {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    ],\n    \"trace_span_id\": \"string\",\n    \"trace_id\": \"string\"\n  }\n}",
)

Response

{
  "ui_results_url": "string"
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}