JudgmentApi
One method per public API route, named after the matching MCP tool.
addExamplesToDataset()
Insert examples to offline-test dataset (POST /v1/projects/{projectId}/datasets/{datasetIdentifier}/examples).
async function addExamplesToDataset(init: { params: { path: { projectId: string; datasetIdentifier: string; } }; body: { examples } }): Promise<{ example_ids: string[]; version_added: number; }>Parameters
init{ params: { path: { projectId: string; datasetIdentifier: string; } }; body: { examples } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; datasetIdentifier: string; } }; body: { examples } }Returns
Promise<{ example_ids: string[]; version_added: number; }>
addTestCases()
Add cases to a test suite’s draft (POST /v1/projects/{projectId}/test-suites/{testSuiteId}/cases).
async function addTestCases(init: { params: { path: { projectId: string; testSuiteId: string; } }; body: { cases } }): Promise<{ case_ids: string[]; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } }; body: { cases } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } }; body: { cases } }Returns
Promise<{ case_ids: string[]; }>
cancelTestRun()
Cancel test run (POST /v1/projects/{projectId}/test-runs/{testRunId}/cancel).
async function cancelTestRun(init: { params: { path: { projectId: string; testRunId: string; } } }): Promise<{ test_run }>Parameters
init{ params: { path: { projectId: string; testRunId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testRunId: string; } } }Returns
Promise<{ test_run }>
createArtifact()
Create artifact (POST /v1/projects/{projectId}/artifacts).
async function createArtifact(init: { params: { path: { projectId: string; } }; body: { scope, name, filename, source_code } }): Promise<{ revision, artifact }>Parameters
init{ params: { path: { projectId: string; } }; body: { scope, name, filename, source_code } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { scope, name, filename, source_code } }Returns
Promise<{ revision, artifact }>
createDataset()
Create offline-test dataset (POST /v1/projects/{projectId}/datasets).
async function createDataset(init: { params: { path: { projectId: string; } }; body: { name, schema, examples?, overwrite, dataset_kind?, metadata? } }): Promise<{ detail, dataset_id, dataset }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, schema, examples?, overwrite, dataset_kind?, metadata? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, schema, examples?, overwrite, dataset_kind?, metadata? } }Returns
Promise<{ detail, dataset_id, dataset }>
createIssue()
Create issue (POST /v1/projects/{projectId}/issues).
async function createIssue(init: { params: { path: { projectId: string; } }; body: { issue_key?, title, diagnosis, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items } }): Promise<{ issue: { id: string; issue_key: string; }; }>Parameters
init{ params: { path: { projectId: string; } }; body: { issue_key?, title, diagnosis, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { issue_key?, title, diagnosis, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items } }Returns
Promise<{ issue: { id: string; issue_key: string; }; }>
createJudge()
Create judge (POST /v1/projects/{projectId}/judges).
async function createJudge(init: { params: { path: { projectId: string; } }; body: { judge } }): Promise<{ judge_id: string; judge_version_id: string; implementation_id: string; }>Parameters
init{ params: { path: { projectId: string; } }; body: { judge } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { judge } }Returns
Promise<{ judge_id: string; judge_version_id: string; implementation_id: string; }>
createMapper()
Create attribute mapper (POST /v1/projects/{projectId}/attribute-mappers).
async function createMapper(init: { params: { path: { projectId: string; } }; body: { name, rules, enabled?, override?, remove_source? } }): Promise<{ mapper }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, rules, enabled?, override?, remove_source? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, rules, enabled?, override?, remove_source? } }Returns
Promise<{ mapper }>
createProject()
Add new project (POST /v1/projects).
async function createProject(init: { body: { project_name } }): Promise<{ project, project_id }>Parameters
init{ body: { project_name } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ body: { project_name } }Returns
Promise<{ project, project_id }>
createScheduledTask()
Create a scheduled task (POST /v1/projects/{projectId}/scheduled-tasks).
async function createScheduledTask(init: { params: { path: { projectId: string; } }; body: { name, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }): Promise<{ task }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }Returns
Promise<{ task }>
createSupabaseArtifactRevision()
Create artifact revision (POST /v1/projects/{projectId}/artifacts/{artifactId}/revisions).
async function createSupabaseArtifactRevision(init: { params: { path: { projectId: string; artifactId: string; } }; body: { version, filename, source_code } }): Promise<{ revision }>Parameters
init{ params: { path: { projectId: string; artifactId: string; } }; body: { version, filename, source_code } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; artifactId: string; } }; body: { version, filename, source_code } }Returns
Promise<{ revision }>
createTestConfig()
Create test config (POST /v1/projects/{projectId}/test-configs).
async function createTestConfig(init: { params: { path: { projectId: string; } }; body: { name, description?, dataset_id?, dataset_name?, judge_ids?, judges? } }): Promise<{ test_config }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, description?, dataset_id?, dataset_name?, judge_ids?, judges? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, description?, dataset_id?, dataset_name?, judge_ids?, judges? } }Returns
Promise<{ test_config }>
createTestSuite()
Create a test suite (POST /v1/projects/{projectId}/test-suites).
async function createTestSuite(init: { params: { path: { projectId: string; } }; body: { name, description?, labels? } }): Promise<{ test_suite_id, name, description, labels, version, created_at, updated_at }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, description?, labels? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, description?, labels? } }Returns
Promise<{ test_suite_id, name, description, labels, version, created_at, updated_at }>
createTestSuiteDraft()
Copy the published cases into the draft to edit (POST /v1/projects/{projectId}/test-suites/{testSuiteId}/draft).
async function createTestSuiteDraft(init: { params: { path: { projectId: string; testSuiteId: string; } } }): Promise<{ version: number; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } } }Returns
Promise<{ version: number; }>
deleteDataset()
Delete offline-test dataset (DELETE /v1/projects/{projectId}/datasets/{datasetIdentifier}).
async function deleteDataset(init: { params: { path: { projectId: string; datasetIdentifier: string; } } }): Promise<{ status: string; message: string; }>Parameters
init{ params: { path: { projectId: string; datasetIdentifier: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; datasetIdentifier: string; } } }Returns
Promise<{ status: string; message: string; }>
deleteJudge()
Delete judge (DELETE /v1/projects/{projectId}/judges/{judgeId}).
async function deleteJudge(init: { params: { path: { projectId: string; judgeId: string; } } }): Promise<{ deleted_count: number; }>Parameters
init{ params: { path: { projectId: string; judgeId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; } } }Returns
Promise<{ deleted_count: number; }>
deleteMapper()
Delete attribute mapper (DELETE /v1/projects/{projectId}/attribute-mappers/{configId}).
async function deleteMapper(init: { params: { path: { projectId: string; configId: string; } } }): Promise<{ success: boolean; }>Parameters
init{ params: { path: { projectId: string; configId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; configId: string; } } }Returns
Promise<{ success: boolean; }>
deleteTestConfig()
Delete test config (DELETE /v1/projects/{projectId}/test-configs/{testConfigId}).
async function deleteTestConfig(init: { params: { path: { projectId: string; testConfigId: string; } } }): Promise<{ status: string; message: string; }>Parameters
init{ params: { path: { projectId: string; testConfigId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testConfigId: string; } } }Returns
Promise<{ status: string; message: string; }>
deleteTestSuite()
Delete a test suite with its runs (DELETE /v1/projects/{projectId}/test-suites/{testSuiteId}).
async function deleteTestSuite(init: { params: { path: { projectId: string; testSuiteId: string; } } }): Promise<{ test_suite_id: string; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } } }Returns
Promise<{ test_suite_id: string; }>
discardTestSuiteDraft()
Throw away the draft’s changes (POST /v1/projects/{projectId}/test-suites/{testSuiteId}/discard).
async function discardTestSuiteDraft(init: { params: { path: { projectId: string; testSuiteId: string; } } }): Promise<{ discarded: boolean; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } } }Returns
Promise<{ discarded: boolean; }>
discoverSchema()
Discover the virtual SQL schema (GET /v1/sql/schema).
async function discoverSchema(init?: FetchOptions): Promise<{ schema: string; }>Parameters
init?FetchOptions
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
FetchOptionsReturns
Promise<{ schema: string; }>
evaluateAllTraces()
Evaluate recent traces (POST /v1/projects/{projectId}/traces/evaluate-all).
async function evaluateAllTraces(init: { params: { path: { projectId: string; } }; body: { judge_names?, limit? } }): Promise<{ enqueued: boolean; limit: number; }>Parameters
init{ params: { path: { projectId: string; } }; body: { judge_names?, limit? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { judge_names?, limit? } }Returns
Promise<{ enqueued: boolean; limit: number; }>
evaluateTraces()
Evaluate stored traces (POST /v1/projects/{projectId}/traces/evaluate).
async function evaluateTraces(init: { params: { path: { projectId: string; } }; body: { trace_ids, judge_names? } }): Promise<{ enqueued: boolean; trace_count: number; }>Parameters
init{ params: { path: { projectId: string; } }; body: { trace_ids, judge_names? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { trace_ids, judge_names? } }Returns
Promise<{ enqueued: boolean; trace_count: number; }>
getArtifactRevision()
Get artifact revision (GET /v1/projects/{projectId}/artifact-revisions).
async function getArtifactRevision(init: { params: { path: { projectId: string; }; query?: { revision_id?: string; artifact_id?: string; version?: number; } } }): Promise<{ revision }>Parameters
init{ params: { path: { projectId: string; }; query?: { revision_id?: string; artifact_id?: string; version?: number; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { revision_id?: string; artifact_id?: string; version?: number; } } }Returns
Promise<{ revision }>
getConversationDisplayRules()
Get conversation display rules (GET /v1/projects/{projectId}/conversation-display-rules).
async function getConversationDisplayRules(init: { params: { path: { projectId: string; } } }): Promise<{ rules: { pattern: string; replacement: string; }[]; }>Parameters
init{ params: { path: { projectId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } } }Returns
Promise<{ rules: { pattern: string; replacement: string; }[]; }>
getDataset()
Fetch dataset page (GET /v1/projects/{projectId}/datasets/{datasetIdentifier}/page).
async function getDataset(init: { params: { path: { projectId: string; datasetIdentifier: string; }; query?: { version?: number; limit?: number; cursor_created_at?: string; cursor_example_id?: string; } } }): Promise<{ dataset, entries, metadata }>Parameters
init{ params: { path: { projectId: string; datasetIdentifier: string; }; query?: { version?: number; limit?: number; cursor_created_at?: string; cursor_example_id?: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; datasetIdentifier: string; }; query?: { version?: number; limit?: number; cursor_created_at?: string; cursor_example_id?: string; } } }Returns
Promise<{ dataset, entries, metadata }>
getIssue()
Get issue (GET /v1/projects/{projectId}/issues/{issueId}).
async function getIssue(init: { params: { path: { projectId: string; issueId: string; } } }): Promise<{ issue }>Parameters
init{ params: { path: { projectId: string; issueId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; issueId: string; } } }Returns
Promise<{ issue }>
getJudge()
Get judge (POST /v1/projects/{projectId}/judges/{judgeId}/lookup).
async function getJudge(init: { params: { path: { projectId: string; judgeId: string; } }; body: { include_code_url?, version? } }): Promise<{ optimistic_eval_eligibility, judge }>Parameters
init{ params: { path: { projectId: string; judgeId: string; } }; body: { include_code_url?, version? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; } }; body: { include_code_url?, version? } }Returns
Promise<{ optimistic_eval_eligibility, judge }>
getJudgeLabel()
Get judge label (GET /v1/projects/{projectId}/behaviors/{behaviorId}/analytics).
async function getJudgeLabel(init: { params: { path: { projectId: string; behaviorId: string; }; query?: { start_date?: string; end_date?: string; } } }): Promise<{ start_time, end_time, granularity, data, summaries }>Parameters
init{ params: { path: { projectId: string; behaviorId: string; }; query?: { start_date?: string; end_date?: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; behaviorId: string; }; query?: { start_date?: string; end_date?: string; } } }Returns
Promise<{ start_time, end_time, granularity, data, summaries }>
getMapper()
Get attribute mapper (GET /v1/projects/{projectId}/attribute-mappers/{configId}).
async function getMapper(init: { params: { path: { projectId: string; configId: string; } } }): Promise<{ mapper }>Parameters
init{ params: { path: { projectId: string; configId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; configId: string; } } }Returns
Promise<{ mapper }>
getScheduledTask()
Get a scheduled task (GET /v1/projects/{projectId}/scheduled-tasks/{taskId}).
async function getScheduledTask(init: { params: { path: { projectId: string; taskId: string; } } }): Promise<{ task }>Parameters
init{ params: { path: { projectId: string; taskId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; taskId: string; } } }Returns
Promise<{ task }>
getTestRun()
Fetch test run (GET /v1/projects/{projectId}/test-runs/{testRunId}).
async function getTestRun(init: { params: { path: { projectId: string; testRunId: string; } } }): Promise<{ test_run, ui_results_url? }>Parameters
init{ params: { path: { projectId: string; testRunId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testRunId: string; } } }Returns
Promise<{ test_run, ui_results_url? }>
getTestRunItems()
Fetch test run items (GET /v1/projects/{projectId}/test-runs/{testRunId}/items).
async function getTestRunItems(init: { params: { path: { projectId: string; testRunId: string; }; query?: { limit?: number; cursor?: string; } } }): Promise<{ results, has_more, next_cursor, ui_results_url? }>Parameters
init{ params: { path: { projectId: string; testRunId: string; }; query?: { limit?: number; cursor?: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testRunId: string; }; query?: { limit?: number; cursor?: string; } } }Returns
Promise<{ results, has_more, next_cursor, ui_results_url? }>
getTestSuiteRunResults()
Get test run results (GET /v1/projects/{projectId}/test-suites/runs/{runId}/results).
async function getTestSuiteRunResults(init: { params: { path: { projectId: string; runId: string; } } }): Promise<{ results }>Parameters
init{ params: { path: { projectId: string; runId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; runId: string; } } }Returns
Promise<{ results }>
getTraceHealth()
Get trace health for one trace (GET /v1/projects/{projectId}/trace-health/traces/{traceId}).
async function getTraceHealth(init: { params: { path: { projectId: string; traceId: string; }; query?: { category?: "trace_health" | "conversation_view"; } } }): Promise<{ trace_health }>Parameters
init{ params: { path: { projectId: string; traceId: string; }; query?: { category?: "trace_health" | "conversation_view"; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; traceId: string; }; query?: { category?: "trace_health" | "conversation_view"; } } }Returns
Promise<{ trace_health }>
getTraceHealthSummary()
Get trace health summary (GET /v1/projects/{projectId}/trace-health/summary).
async function getTraceHealthSummary(init: { params: { path: { projectId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; } } }): Promise<{ summary }>Parameters
init{ params: { path: { projectId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; } } }Returns
Promise<{ summary }>
listDatasets()
Fetch offline-test datasets (GET /v1/projects/{projectId}/datasets).
async function listDatasets(init: { params: { path: { projectId: string; } } }): Promise<{ dataset_id, organization_id?, project_id?, name, schema?, current_version?, created_at?, updated_at?, creat...Parameters
init{ params: { path: { projectId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } } }Returns
Promise<{ dataset_id, organization_id?, project_id?, name, schema?, current_version?, created_at?, updated_at?, creat...
listIssues()
List issues (GET /v1/projects/{projectId}/issues).
async function listIssues(init: { params: { path: { projectId: string; }; query?: { user_status?: "needs_review" | "ignored" | "resolved" | "merged" | "monitored"; production_activity?: "new" | "ongoing" | "increasing" | "recurring"; priority?: "low" | "medium" | "high" | "critical"; } } }): Promise<{ issues }>Parameters
init{ params: { path: { projectId: string; }; query?: { user_status?: "needs_review" | "ignored" | "resolved" | "merged" | "monitored"; production_activity?: "new" | "ongoing" | "increasing" | "recurring"; priority?: "low" | "medium" | "high" | "critical"; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { user_status?: "needs_review" | "ignored" | "resolved" | "merged" | "monitored"; production_activity?: "new" | "ongoing" | "increasing" | "recurring"; priority?: "low" | "medium" | "high" | "critical"; } } }Returns
Promise<{ issues }>
listJudgeActionGroups()
List judge action groups (GET /v1/projects/{projectId}/judges/action-groups).
async function listJudgeActionGroups(init: { params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }): Promise<{ action_groups }>Parameters
init{ params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }Returns
Promise<{ action_groups }>
listJudges()
List judges (GET /v1/projects/{projectId}/judges).
async function listJudges(init: { params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }): Promise<{ judges }>Parameters
init{ params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { version?: "latest" | "production"; } } }Returns
Promise<{ judges }>
listMappers()
List attribute mappers (GET /v1/projects/{projectId}/attribute-mappers).
async function listMappers(init: { params: { path: { projectId: string; } } }): Promise<{ mappers }>Parameters
init{ params: { path: { projectId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } } }Returns
Promise<{ mappers }>
listMemoryEntries()
List memory entries (GET /v1/projects/{projectId}/memory/entries).
async function listMemoryEntries(init: { params: { path: { projectId: string; } } }): Promise<{ entries: { id: string; name: string; type: "folder" | "file"; created_at: string; updated_at: string; }[]; }>Parameters
init{ params: { path: { projectId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } } }Returns
Promise<{ entries: { id: string; name: string; type: "folder" | "file"; created_at: string; updated_at: string; }[]; }>
listOrganizations()
List organization memberships (GET /v1/organizations/memberships).
async function listOrganizations(init?: FetchOptions): Promise<{ organizations }>Parameters
init?FetchOptions
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
FetchOptionsReturns
Promise<{ organizations }>
listProjects()
List project summaries (GET /v1/projects/summaries).
async function listProjects(init?: FetchOptions): Promise<{ projects }>Parameters
init?FetchOptions
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
FetchOptionsReturns
Promise<{ projects }>
listScheduledTasks()
List scheduled tasks (GET /v1/projects/{projectId}/scheduled-tasks).
async function listScheduledTasks(init: { params: { path: { projectId: string; } } }): Promise<{ tasks }>Parameters
init{ params: { path: { projectId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } } }Returns
Promise<{ tasks }>
listTestCases()
List test cases in a version or the draft (GET /v1/projects/{projectId}/test-suites/{testSuiteId}/cases).
async function listTestCases(init: { params: { path: { projectId: string; testSuiteId: string; }; query?: { version?: string; offset?: number; limit?: number; } } }): Promise<{ version, cases, next_offset }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; }; query?: { version?: string; offset?: number; limit?: number; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; }; query?: { version?: string; offset?: number; limit?: number; } } }Returns
Promise<{ version, cases, next_offset }>
listTestConfigs()
Fetch test configs (GET /v1/projects/{projectId}/test-configs).
async function listTestConfigs(init: { params: { path: { projectId: string; }; query?: { dataset_id?: string; } } }): Promise<{ test_configs }>Parameters
init{ params: { path: { projectId: string; }; query?: { dataset_id?: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { dataset_id?: string; } } }Returns
Promise<{ test_configs }>
listTestRuns()
Fetch test runs (GET /v1/projects/{projectId}/test-runs).
async function listTestRuns(init: { params: { path: { projectId: string; }; query?: { test_config_id?: string; dataset_id?: string; status?: "pending" | "running" | "completed" | "error" | "cancelled"; limit?: number; } } }): Promise<{ test_runs }>Parameters
init{ params: { path: { projectId: string; }; query?: { test_config_id?: string; dataset_id?: string; status?: "pending" | "running" | "completed" | "error" | "cancelled"; limit?: number; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { test_config_id?: string; dataset_id?: string; status?: "pending" | "running" | "completed" | "error" | "cancelled"; limit?: number; } } }Returns
Promise<{ test_runs }>
listTestSuiteRuns()
List test runs (GET /v1/projects/{projectId}/test-suites/{testSuiteId}/runs).
async function listTestSuiteRuns(init: { params: { path: { projectId: string; testSuiteId: string; }; query?: { offset?: number; limit?: number; } } }): Promise<{ runs, next_offset }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; }; query?: { offset?: number; limit?: number; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; }; query?: { offset?: number; limit?: number; } } }Returns
Promise<{ runs, next_offset }>
listTestSuites()
List test suites (GET /v1/projects/{projectId}/test-suites).
async function listTestSuites(init: { params: { path: { projectId: string; }; query?: { offset?: number; limit?: number; } } }): Promise<{ test_suites, next_offset }>Parameters
init{ params: { path: { projectId: string; }; query?: { offset?: number; limit?: number; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; }; query?: { offset?: number; limit?: number; } } }Returns
Promise<{ test_suites, next_offset }>
listTraceHealthFailures()
List traces failing a trace health check (GET /v1/projects/{projectId}/trace-health/checks/{checkId}/traces).
async function listTraceHealthFailures(init: { params: { path: { projectId: string; checkId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; limit?: number; cursor_trace_started_at?: string; cursor_trace_id?: string; } } }): Promise<{ traces, has_more, next_cursor }>Parameters
init{ params: { path: { projectId: string; checkId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; limit?: number; cursor_trace_started_at?: string; cursor_trace_id?: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; checkId: string; }; query: { category?: "trace_health" | "conversation_view"; start_time: string; end_time: string; limit?: number; cursor_trace_started_at?: string; cursor_trace_id?: string; } } }Returns
Promise<{ traces, has_more, next_cursor }>
publishTestSuite()
Publish the draft as the next version (POST /v1/projects/{projectId}/test-suites/{testSuiteId}/publish).
async function publishTestSuite(init: { params: { path: { projectId: string; testSuiteId: string; } } }): Promise<{ version: number; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } } }Returns
Promise<{ version: number; }>
readMemoryFiles()
Fetch memory files (POST /v1/projects/{projectId}/memory/files/fetch).
async function readMemoryFiles(init: { params: { path: { projectId: string; } }; body: { ids?, names? } }): Promise<{ files }>Parameters
init{ params: { path: { projectId: string; } }; body: { ids?, names? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { ids?, names? } }Returns
Promise<{ files }>
removeExamplesFromDataset()
Remove examples from offline-test dataset (POST /v1/projects/{projectId}/datasets/{datasetIdentifier}/examples/remove).
async function removeExamplesFromDataset(init: { params: { path: { projectId: string; datasetIdentifier: string; } }; body: { example_ids } }): Promise<{ removed_example_ids: string[]; version_created: number; }>Parameters
init{ params: { path: { projectId: string; datasetIdentifier: string; } }; body: { example_ids } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; datasetIdentifier: string; } }; body: { example_ids } }Returns
Promise<{ removed_example_ids: string[]; version_created: number; }>
removeTestCase()
Remove a case from the draft (DELETE /v1/projects/{projectId}/test-suites/{testSuiteId}/cases/{caseId}).
async function removeTestCase(init: { params: { path: { projectId: string; testSuiteId: string; caseId: string; } } }): Promise<{ case_id: string; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; caseId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; caseId: string; } } }Returns
Promise<{ case_id: string; }>
runTest()
Prepare and queue test run (POST /v1/projects/{projectId}/test-runs).
async function runTest(init: { params: { path: { projectId: string; } }; body: { test_run_id?, test_config_id, name?, dataset_version_id?, dataset_version_number?, judge_versions?, source?, agent_errors?, agent_traces? } }): Promise<{ test_run, dataset, dataset_version, examples, judges, evaluation_runs, ui_results_url }>Parameters
init{ params: { path: { projectId: string; } }; body: { test_run_id?, test_config_id, name?, dataset_version_id?, dataset_version_number?, judge_versions?, source?, agent_errors?, agent_traces? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { test_run_id?, test_config_id, name?, dataset_version_id?, dataset_version_number?, judge_versions?, source?, agent_errors?, agent_traces? } }Returns
Promise<{ test_run, dataset, dataset_version, examples, judges, evaluation_runs, ui_results_url }>
scoreTestRun()
Score a test run (POST /v1/projects/{projectId}/test-suites/runs/{runId}/score).
async function scoreTestRun(init: { params: { path: { projectId: string; runId: string; } } }): Promise<{ run_id: string; }>Parameters
init{ params: { path: { projectId: string; runId: string; } } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; runId: string; } } }Returns
Promise<{ run_id: string; }>
searchMemoryFiles()
Search memory files (POST /v1/projects/{projectId}/memory/search).
async function searchMemoryFiles(init: { params: { path: { projectId: string; } }; body: { query, limit? } }): Promise<{ results }>Parameters
init{ params: { path: { projectId: string; } }; body: { query, limit? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { query, limit? } }Returns
Promise<{ results }>
setConversationDisplayRules()
Replace conversation display rules (PUT /v1/projects/{projectId}/conversation-display-rules).
async function setConversationDisplayRules(init: { params: { path: { projectId: string; } }; body: { rules } }): Promise<{ rules: { pattern: string; replacement: string; }[]; }>Parameters
init{ params: { path: { projectId: string; } }; body: { rules } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { rules } }Returns
Promise<{ rules: { pattern: string; replacement: string; }[]; }>
setJudgeActionGroups()
Set judge action groups (POST /v1/projects/{projectId}/judges/{judgeId}/action-groups).
async function setJudgeActionGroups(init: { params: { path: { projectId: string; judgeId: string; } }; body: { source_version, target_version, promote_to_production, action_groups } }): Promise<{ judge_id: string; judge_version_id: string | null; }>Parameters
init{ params: { path: { projectId: string; judgeId: string; } }; body: { source_version, target_version, promote_to_production, action_groups } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; } }; body: { source_version, target_version, promote_to_production, action_groups } }Returns
Promise<{ judge_id: string; judge_version_id: string | null; }>
setJudgeOptimisticEvalEnabled()
Set judge version optimistic eval (PUT /v1/projects/{projectId}/judges/{judgeId}/versions/{judgeVersionId}/optimistic-eval).
async function setJudgeOptimisticEvalEnabled(init: { params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { enabled } }): Promise<{ judge_id: string; judge_version_id: string; optimistic_eval_enabled: boolean; }>Parameters
init{ params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { enabled } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { enabled } }Returns
Promise<{ judge_id: string; judge_version_id: string; optimistic_eval_enabled: boolean; }>
setJudgePreprocessorArtifactRevision()
Set judge version preprocessing artifact (PUT /v1/projects/{projectId}/judges/{judgeId}/versions/{judgeVersionId}/preprocessor-artifact-revision).
async function setJudgePreprocessorArtifactRevision(init: { params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { preprocessor_artifact_revision_id } }): Promise<{ judge_id, judge_version_id, preprocessor_artifact_revision_id, optimistic_eval_enabled }>Parameters
init{ params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { preprocessor_artifact_revision_id } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; judgeVersionId: string; } }; body: { preprocessor_artifact_revision_id } }Returns
Promise<{ judge_id, judge_version_id, preprocessor_artifact_revision_id, optimistic_eval_enabled }>
sqlQuery()
Run a tenant-scoped virtual SQL query (POST /v1/projects/{projectId}/sql).
async function sqlQuery(init: { params: { path: { projectId: string; } }; body: { sql } }): Promise<{ catalog_version, columns, rows, row_count, elapsed_ms }>Parameters
init{ params: { path: { projectId: string; } }; body: { sql } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { sql } }Returns
Promise<{ catalog_version, columns, rows, row_count, elapsed_ms }>
startTestSuiteRun()
Start a test run (POST /v1/projects/{projectId}/test-suites/{testSuiteId}/runs).
async function startTestSuiteRun(init: { params: { path: { projectId: string; testSuiteId: string; } }; body: { case_filter?, candidate?, candidate_meta?, trigger } }): Promise<{ run_id, test_suite_version, cases }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; } }; body: { case_filter?, candidate?, candidate_meta?, trigger } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; } }; body: { case_filter?, candidate?, candidate_meta?, trigger } }Returns
Promise<{ run_id, test_suite_version, cases }>
testOptimisticEvalCandidate()
Test optimistic eval candidate (POST /v1/projects/{projectId}/artifacts/optimistic-eval-tests).
async function testOptimisticEvalCandidate(init: { params: { path: { projectId: string; } }; body: { trace_id, source_code, judge_id, judge_version_id } }): Promise<{ judge_id, judge_version_id, judge_major_version, judge_minor_version, score_type, trace_id, result, ordinar...Parameters
init{ params: { path: { projectId: string; } }; body: { trace_id, source_code, judge_id, judge_version_id } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { trace_id, source_code, judge_id, judge_version_id } }Returns
Promise<{ judge_id, judge_version_id, judge_major_version, judge_minor_version, score_type, trace_id, result, ordinar...
testPreprocessingCandidate()
Test preprocessing candidate (POST /v1/projects/{projectId}/artifacts/preprocessing-tests).
async function testPreprocessingCandidate(init: { params: { path: { projectId: string; } }; body: { trace_id, source_code } }): Promise<{ result }>Parameters
init{ params: { path: { projectId: string; } }; body: { trace_id, source_code } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { trace_id, source_code } }Returns
Promise<{ result }>
updateIssue()
Update issue (PATCH /v1/projects/{projectId}/issues/{issueId}).
async function updateIssue(init: { params: { path: { projectId: string; issueId: string; } }; body: { title?, diagnosis?, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items?, user_status?, judge_id?, behavior_id?, message? } }): Promise<{ issue }>Parameters
init{ params: { path: { projectId: string; issueId: string; } }; body: { title?, diagnosis?, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items?, user_status?, judge_id?, behavior_id?, message? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; issueId: string; } }; body: { title?, diagnosis?, impact_summary?, priority?, production_activity?, agent_reasoning?, cited_items?, user_status?, judge_id?, behavior_id?, message? } }Returns
Promise<{ issue }>
updateJudge()
Update judge (PATCH /v1/projects/{projectId}/judges/{judgeId}).
async function updateJudge(init: { params: { path: { projectId: string; judgeId: string; } }; body: { update } }): Promise<{ judge_id: string; judge_version_id: string | null; implementation_id: string | null; }>Parameters
init{ params: { path: { projectId: string; judgeId: string; } }; body: { update } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; judgeId: string; } }; body: { update } }Returns
Promise<{ judge_id: string; judge_version_id: string | null; implementation_id: string | null; }>
updateJudgeLabel()
Update judge label (PATCH /v1/projects/{projectId}/behaviors/{behaviorId}).
async function updateJudgeLabel(init: { params: { path: { projectId: string; behaviorId: string; } }; body: { description? } }): Promise<{ behavior }>Parameters
init{ params: { path: { projectId: string; behaviorId: string; } }; body: { description? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; behaviorId: string; } }; body: { description? } }Returns
Promise<{ behavior }>
updateMapper()
Update attribute mapper (PATCH /v1/projects/{projectId}/attribute-mappers/{configId}).
async function updateMapper(init: { params: { path: { projectId: string; configId: string; } }; body: { name?, rules?, enabled?, override?, remove_source? } }): Promise<{ mapper }>Parameters
init{ params: { path: { projectId: string; configId: string; } }; body: { name?, rules?, enabled?, override?, remove_source? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; configId: string; } }; body: { name?, rules?, enabled?, override?, remove_source? } }Returns
Promise<{ mapper }>
updateScheduledTask()
Update a scheduled task (PATCH /v1/projects/{projectId}/scheduled-tasks/{taskId}).
async function updateScheduledTask(init: { params: { path: { projectId: string; taskId: string; } }; body: { name?, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }): Promise<{ task, message }>Parameters
init{ params: { path: { projectId: string; taskId: string; } }; body: { name?, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; taskId: string; } }; body: { name?, description?, instructions?, mode?, tool_permission_mode?, schedule?, slack_channel_ids? } }Returns
Promise<{ task, message }>
updateTestCase()
Replace a case in the draft (PUT /v1/projects/{projectId}/test-suites/{testSuiteId}/cases/{caseId}).
async function updateTestCase(init: { params: { path: { projectId: string; testSuiteId: string; caseId: string; } }; body: { name?, input, expected?, tags?, metadata? } }): Promise<{ case_id: string; }>Parameters
init{ params: { path: { projectId: string; testSuiteId: string; caseId: string; } }; body: { name?, input, expected?, tags?, metadata? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testSuiteId: string; caseId: string; } }; body: { name?, input, expected?, tags?, metadata? } }Returns
Promise<{ case_id: string; }>
updateTestConfig()
Update test config (PATCH /v1/projects/{projectId}/test-configs/{testConfigId}).
async function updateTestConfig(init: { params: { path: { projectId: string; testConfigId: string; } }; body: { name?, description? } }): Promise<{ test_config }>Parameters
init{ params: { path: { projectId: string; testConfigId: string; } }; body: { name?, description? } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; testConfigId: string; } }; body: { name?, description? } }Returns
Promise<{ test_config }>
writeMemoryFile()
Write a memory file (PUT /v1/projects/{projectId}/memory/files).
async function writeMemoryFile(init: { params: { path: { projectId: string; } }; body: { name, body } }): Promise<{ file }>Parameters
init{ params: { path: { projectId: string; } }; body: { name, body } }
params fills the URL path and query string. body lists the JSON body fields; the REST API reference has their full schema. Also accepts fetch options such as headers and signal.
{ params: { path: { projectId: string; } }; body: { name, body } }Returns
Promise<{ file }>