Judgment Labs Logo
DocumentationSDK ReferenceInteractive DemoChangelog
Judgeval Python SDKResponse Types

ScorerData

Individual scorer result containing score, reasoning, and metadata

Individual scorer result containing the score, reasoning, and metadata for a single scorer applied to an example.

namerequired

:str

Name of the scorer that generated this result

thresholdrequired

:float

Threshold value used to determine pass/fail for this scorer

successrequired

:bool

Whether this individual scorer succeeded (score >= threshold)

score

:float

Numerical score returned by the scorer (typically 0.0-1.0)

reason

:str

Human-readable explanation of why the scorer gave this result

id

:str
Unique identifier for this scorer instance

strict_mode

:bool
Whether the scorer was run in strict mode

evaluation_model

:Union[List[str], str]

Model(s) used for evaluation (e.g., "gpt-4", ["gpt-4", "claude-3"])

error

:str

Error message if the scorer failed to execute

additional_metadata

:Dict[str, Any]

Extra information specific to this scorer or evaluation run

Usage Examples

# Access scorer data from a ScoringResult
scoring_result = client.evaluate(examples=[example], scorers=[faithfulness_scorer])[0]

for scorer_data in scoring_result.scorers_data:
    print(f"Scorer: {scorer_data.name}")
    print(f"Score: {scorer_data.score} (threshold: {scorer_data.threshold})")
    print(f"Success: {scorer_data.success}")
    print(f"Reason: {scorer_data.reason}")

    if scorer_data.error:
        print(f"Error: {scorer_data.error}")

ScoringResult

Complete evaluation results for a single example

DatasetInfo

Information about a dataset after creation or retrieval operations

Last updated on

On this page

Usage Examples