EvaluationRunOptions

Properties

PropType

examples?Example[]

The examples to evaluate.

TypeExample[]

scorers?string[] | Judge<BaseResponse>[]

Hosted scorer names (strings like "faithfulness") or custom Judge instances. Cannot mix both.

Typestring[] | Judge<BaseResponse>[]

evalRunName?string

A name for this run, visible in the dashboard.

Typestring

assertTest?boolean | undefined

If true, throws an error when any scorer fails its threshold. Useful in CI/CD pipelines.

Typeboolean | undefined

timeoutSeconds?number | undefined

Maximum seconds to wait for hosted scorer results before timing out.

Typenumber | undefined