LocalEvaluatorRunner
run()
Execute an evaluation run and return results.
def run(examples, scorers, eval_run_name, assert_test=False, timeout_seconds=300) -> typing.List:Parameters
PropType
examplesList[Example]
Examples to evaluate.
Type
List[Example]scorersList[S]
Scorers to run (strings or Judge instances).
Type
List[S]eval_run_namestr
Name for this evaluation run.
Type
strassert_test?bool
Deprecated and ignored by the current evaluation result payload.
Type
boolDefault
Falsetimeout_seconds?int
Maximum time to wait for results.
Type
intDefault
300Returns
typing.List - A list of ScoringResult objects, one per example.