Log evaluation results

Log evaluation results from SDK

POST/v1/projects/{projectId}/eval-results/examples

Authorization

AuthorizationPersonal API key · headerrequired

Your personal API key from Settings > Personal API Keys, sent as Bearer <key>.

X-Organization-IdOrganization ID · headerrequired

The organization to act in. Copy it with Copy organization ID in the organization menu or the settings sidebar.

Path parameters

projectIdstringrequired

Request body

requiredapplication/json

resultsOfflineTestsLocalScorerResult[]required

Show properties

Array of OfflineTestsLocalScorerResult

scorers_dataobject[]required

Show properties

Array of object

scorer_namestringrequired

valueboolean | number | stringrequired

Show properties

Any of:

boolean

boolean

number

number

string

string

reasonstringrequired

citationsobject[] | null

Show properties

Any of:

object[]

Array of object

span_idstringrequired

span_attributestringrequired

null

null

errorstring | null

Show properties

Any of:

string

string

null

null

data_objectExamplerequired

Show properties

example_idstringrequired

created_atstringrequired

namestring | null

Show properties

Any of:

string

string

null

null

conversation_idstring | null

Show properties

Any of:

string

string

null

null

trace_idstring | null

Show properties

Any of:

string

string

null

null

span_idstring | null

Show properties

Any of:

string

string

null

null

offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

session_idstring | null

Show properties

Any of:

string

string

null

null

session_trace_idsstring[]

agent_offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

runExampleEvaluationRunrequired

Show properties

idstring | null

Show properties

Any of:

string

string

null

null

project_idstringrequired

eval_namestringrequired

modelstring | null

Show properties

Any of:

string

string

null

null

created_atstring | null

Show properties

Any of:

string

string

null

null

user_idstring

scorersany[] | null

Show properties

Any of:

any[]

Array of any

any

null

null

custom_scorersBaseScorer[]required

default: []

Show properties

Array of BaseScorer

score_typestringrequired

namestringrequired

class_namestring | null

Show properties

Any of:

string

string

null

null

scorenumber | null

Show properties

Any of:

number

number

null

null

minimum_score_rangenumber

default: 0

maximum_score_rangenumber

default: 1

score_breakdownobject | null

Show properties

Any of:

object

object

null

null

reasonstring | object

Show properties

Any of:

string

string

object

object

successboolean | null

Show properties

Any of:

boolean

boolean

null

null

modelstring | null

Show properties

Any of:

string

string

null

null

errorstring | null

Show properties

Any of:

string

string

null

null

additional_metadataobject | null

Show properties

Any of:

object

object

null

null

userstring | null

Show properties

Any of:

string

string

null

null

server_hostedboolean | null

Show properties

Any of:

boolean

boolean

null

null

using_native_modelboolean | null

Show properties

Any of:

boolean

boolean

null

null

required_paramsstring[] | null

Show properties

Any of:

string[]

Array of string

string

null

null

strict_modeboolean | null

Show properties

Any of:

boolean

boolean

null

null

judgment_scorersScorerConfig[]required

default: []

Show properties

Array of ScorerConfig

score_typestring | null

Show properties

Any of:

string

string

null

null

namestringrequired

modelstring | null

Show properties

Any of:

string

string

null

null

required_paramsstring[] | null

Show properties

Any of:

string[]

Array of string

string

null

null

kwargsobject | null

Show properties

Any of:

object

object

null

null

result_typestring

Allowed:numericbinarycategorical

examplesExample[]required

min items 1

Show properties

Array of Example

example_idstringrequired

created_atstringrequired

namestring | null

Show properties

Any of:

string

string

null

null

conversation_idstring | null

Show properties

Any of:

string

string

null

null

trace_idstring | null

Show properties

Any of:

string

string

null

null

span_idstring | null

Show properties

Any of:

string

string

null

null

offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

session_idstring | null

Show properties

Any of:

string

string

null

null

session_trace_idsstring[]

agent_offline_trace_idstring | null

Show properties

Any of:

string

string

null

null

trace_span_idstring | null

Show properties

Any of:

string

string

null

null

trace_idstring | null

Show properties

Any of:

string

string

null

null

Responses

200

Response for status 200

ui_results_urlstringrequired

422

Response for status 422

errorstringrequired

messagestring

codestring

detailsstring[]

Try it

Server

Base URLCustom base URLOverrides the selected server when set.

Authorization

Personal API keyOrganization IDRemember on this deviceInclude my values in samples

Parameters

projectIdpath*

Bodyapplication/json

Request body

{
  "results": [
    {
      "scorers_data": [
        {
          "scorer_name": "string",
          "value": true,
          "reason": "string",
          "citations": [
            {
              "span_id": "string",
              "span_attribute": "string"
            }
          ],
          "error": "string"
        }
      ],
      "data_object": {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    }
  ],
  "run": {
    "id": "string",
    "project_id": "string",
    "eval_name": "string",
    "model": "string",
    "created_at": "string",
    "user_id": "string",
    "scorers": [
      null
    ],
    "custom_scorers": [],
    "judgment_scorers": [],
    "examples": [
      {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    ],
    "trace_span_id": "string",
    "trace_id": "string"
  }
}

Request

curl -X POST 'https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results/examples' \
  -H 'Authorization: Bearer YOUR_API_KEY' \
  -H 'X-Organization-Id: YOUR_ORGANIZATION_ID' \
  -H 'Accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{
  "results": [
    {
      "scorers_data": [
        {
          "scorer_name": "string",
          "value": true,
          "reason": "string",
          "citations": [
            {
              "span_id": "string",
              "span_attribute": "string"
            }
          ],
          "error": "string"
        }
      ],
      "data_object": {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    }
  ],
  "run": {
    "id": "string",
    "project_id": "string",
    "eval_name": "string",
    "model": "string",
    "created_at": "string",
    "user_id": "string",
    "scorers": [
      null
    ],
    "custom_scorers": [],
    "judgment_scorers": [],
    "examples": [
      {
        "example_id": "string",
        "created_at": "string",
        "name": "string",
        "conversation_id": "string",
        "trace_id": "string",
        "span_id": "string",
        "offline_trace_id": "string",
        "session_id": "string",
        "session_trace_ids": [
          "string"
        ],
        "agent_offline_trace_id": "string"
      }
    ],
    "trace_span_id": "string",
    "trace_id": "string"
  }
}'

const response = await fetch("https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results/examples", {
  method: "POST",
  headers: {
    "Authorization": "Bearer YOUR_API_KEY",
    "X-Organization-Id": "YOUR_ORGANIZATION_ID",
    "Accept": "application/json",
    "Content-Type": "application/json"
  },
  body: "{\n  \"results\": [\n    {\n      \"scorers_data\": [\n        {\n          \"scorer_name\": \"string\",\n          \"value\": true,\n          \"reason\": \"string\",\n          \"citations\": [\n            {\n              \"span_id\": \"string\",\n              \"span_attribute\": \"string\"\n            }\n          ],\n          \"error\": \"string\"\n        }\n      ],\n      \"data_object\": {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    }\n  ],\n  \"run\": {\n    \"id\": \"string\",\n    \"project_id\": \"string\",\n    \"eval_name\": \"string\",\n    \"model\": \"string\",\n    \"created_at\": \"string\",\n    \"user_id\": \"string\",\n    \"scorers\": [\n      null\n    ],\n    \"custom_scorers\": [],\n    \"judgment_scorers\": [],\n    \"examples\": [\n      {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    ],\n    \"trace_span_id\": \"string\",\n    \"trace_id\": \"string\"\n  }\n}"
});

import requests

response = requests.post(
    "https://api.judgmentlabs.ai/v1/projects/YOUR_PROJECT_ID/eval-results/examples",
    headers={
        "Authorization": "Bearer YOUR_API_KEY",
        "X-Organization-Id": "YOUR_ORGANIZATION_ID",
        "Accept": "application/json",
        "Content-Type": "application/json"
    },
    data="{\n  \"results\": [\n    {\n      \"scorers_data\": [\n        {\n          \"scorer_name\": \"string\",\n          \"value\": true,\n          \"reason\": \"string\",\n          \"citations\": [\n            {\n              \"span_id\": \"string\",\n              \"span_attribute\": \"string\"\n            }\n          ],\n          \"error\": \"string\"\n        }\n      ],\n      \"data_object\": {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    }\n  ],\n  \"run\": {\n    \"id\": \"string\",\n    \"project_id\": \"string\",\n    \"eval_name\": \"string\",\n    \"model\": \"string\",\n    \"created_at\": \"string\",\n    \"user_id\": \"string\",\n    \"scorers\": [\n      null\n    ],\n    \"custom_scorers\": [],\n    \"judgment_scorers\": [],\n    \"examples\": [\n      {\n        \"example_id\": \"string\",\n        \"created_at\": \"string\",\n        \"name\": \"string\",\n        \"conversation_id\": \"string\",\n        \"trace_id\": \"string\",\n        \"span_id\": \"string\",\n        \"offline_trace_id\": \"string\",\n        \"session_id\": \"string\",\n        \"session_trace_ids\": [\n          \"string\"\n        ],\n        \"agent_offline_trace_id\": \"string\"\n      }\n    ],\n    \"trace_span_id\": \"string\",\n    \"trace_id\": \"string\"\n  }\n}",
)

Response

{
  "ui_results_url": "string"
}

{
  "error": "string",
  "message": "string",
  "code": "string",
  "details": [
    "string"
  ]
}