LocalEvaluatorRunner
Evaluation runner for custom (in-process) scorers. Runs all Judge instances locally against the provided examples, posts results to the Judgment platform, then polls for finalized scores. Used internally by Evaluation.
Evaluation runner for custom (in-process) scorers.
Runs all Judge instances locally against the provided examples, posts results to the Judgment platform, then polls for finalized scores. Used internally by Evaluation.
EvaluatorRunner
Abstract base for evaluation runners. Provides the shared run -> poll -> display flow. Subclasses implement `_buildPayload` and `_submit` for local vs hosted mode.
HostedEvaluatorRunner
Evaluation runner for hosted (server-side) scorers. Submits scorer names to the Judgment platform's evaluation queue and polls for results. Used internally by Evaluation.
Last updated on