OfflineTestRunner

Executes the offline-test lifecycle for a test config: resolve the dataset version, optionally run the agent to produce offline traces, create the test run, wait for terminal status, fetch results, evaluate the pass condition, and report successes.

runAgent()

Run the agent once per dataset example, producing one offline trace each.

Up to concurrency examples run at a time; the default of 1 runs them sequentially in dataset order. Each example id rides on the OTel context, so the offline span processor can pair it with the root span it starts regardless of completion order.

async function runAgent(agentFunction: AgentFunction, examples: ExampleRow[], concurrency: number = 1): Promise<Record<string, string>>

Parameters

Returns

Promise<Record<string, string>>