TestSuitesFactory

Create test suites and edit their cases. Access via judgeval.testSuites.

Cases change only in a suite’s draft (version + 1). New cases go straight into the draft; to edit or remove published cases, call createDraft first. publish freezes the draft as the next version.

listCases()

Cases in a published version (default: the latest) or the draft.

async function listCases(testSuiteId: string, options: { version?: number | "draft"; offset?: number; limit?: number; } = {}): Promise<Page<"cases", TestCase> & { version: number }>

Parameters

Returns

Promise<Page<"cases", TestCase> & { version: number }>

addCases()

Add cases to the draft. Returns their ids in order.

async function addCases(testSuiteId: string, cases: TestCaseInput[]): Promise<{ case_ids: string[] }>

Parameters

Returns

Promise<{ case_ids: string[] }>

updateCase()

Replace a case in the draft. Call createDraft first for published cases.

async function updateCase(testSuiteId: string, caseId: string, testCase: TestCaseInput): Promise<{ case_id: string }>

Parameters

Returns

Promise<{ case_id: string }>

removeCase()

Remove a case from the draft. Call createDraft first for published cases.

async function removeCase(testSuiteId: string, caseId: string): Promise<{ case_id: string }>

Parameters

Returns

Promise<{ case_id: string }>

createDraft()

Copy the published cases into the draft so they can be edited.

async function createDraft(testSuiteId: string): Promise<{ version: number }>

Parameters

Returns

Promise<{ version: number }>

publish()

Freeze the draft as the next version.

async function publish(testSuiteId: string): Promise<{ version: number }>

Parameters

Returns

Promise<{ version: number }>

discard()

Throw away the draft’s changes.

async function discard(testSuiteId: string): Promise<{ discarded: boolean }>

Parameters

Returns

Promise<{ discarded: boolean }>