EvaluatorRunner
Abstract base for evaluation runners. Provides the shared run -> poll -> display flow. Subclasses implement `_buildPayload` and `_submit` for local vs hosted mode.
Abstract base for evaluation runners.
Provides the shared run -> poll -> display flow.
Subclasses implement _buildPayload and _submit for local vs hosted mode.
AsyncEvaluateOptions
Options for BaseTracer.asyncEvaluate.
LocalEvaluatorRunner
Evaluation runner for custom (in-process) scorers. Runs all Judge instances locally against the provided examples, posts results to the Judgment platform, then polls for finalized scores. Used internally by Evaluation.
Last updated on