TestRunsFactory

Run test suites and read results. Access via judgeval.testRuns.

const run = await judgeval.testRuns.start(testSuiteId, { candidate: process.env.GIT_SHA });
await Promise.all(
  run.cases.map((c) => Tracer.testCase(run.run_id, c.case_id, () => agent(c.input))),
);
const { results } = await judgeval.testRuns.results(run.run_id);

start()

Start a run of the test suite’s latest published version.

async function start(testSuiteId: string, options: { case_filter?: CaseFilter; candidate?: string; candidate_meta?: Record<string, unknown>; trigger?: "manual" | "ci" |... = {}): Promise<StartedRun>

Parameters

Returns

Promise<StartedRun>

score()

Trigger scoring of an existing run’s traces with the latest published version. Returns the new run right away; its cases are scored in the background, so poll get.

async function score(runId: string): Promise<{ run_id: string }>

Parameters

Returns

Promise<{ run_id: string }>