Judgeval Python SDKResponse Types
ScorerData
Individual scorer result containing score, reasoning, and metadata
Individual scorer result containing the score, reasoning, and metadata for a single scorer applied to an example.
namerequired:str
Name of the scorer that generated this result
thresholdrequired:float
Threshold value used to determine pass/fail for this scorer
successrequired:bool
Whether this individual scorer succeeded (score >= threshold)
score:float
Numerical score returned by the scorer (typically 0.0-1.0)
reason:str
Human-readable explanation of why the scorer gave this result
id:str
Unique identifier for this scorer instance
strict_mode:bool
Whether the scorer was run in strict mode
evaluation_model:Union[List[str], str]
Model(s) used for evaluation (e.g., "gpt-4", ["gpt-4", "claude-3"])
error:str
Error message if the scorer failed to execute
additional_metadata:Dict[str, Any]
Extra information specific to this scorer or evaluation run
Usage Examples
# Access scorer data from a ScoringResult
scoring_result = client.evaluate(examples=[example], scorers=[faithfulness_scorer])[0]
for scorer_data in scoring_result.scorers_data:
print(f"Scorer: {scorer_data.name}")
print(f"Score: {scorer_data.score} (threshold: {scorer_data.threshold})")
print(f"Success: {scorer_data.success}")
print(f"Reason: {scorer_data.reason}")
if scorer_data.error:
print(f"Error: {scorer_data.error}")