AgentJudge
A prompt-based Agent judge stored on the Judgment platform.
ScoreType?Any
AnyLiteral['numeric', 'binary', 'categorical']ReasoningLevel?Any
AnyLiteral['lite', 'normal', 'high']A prompt-based Agent judge stored on the Judgment platform.
Agent judges are LLM-driven scorers. The prompt field is the rubric
prompt used by the Agent judge harness when scoring an output. Versions
are managed implicitly — calling .update() writes a new minor version of
the judge (matching the default “save” flow in the UI).
client = Judgeval(project_name="my-project")
judge = client.agent_judges.create(
name="helpfulness",
prompt="Did the assistant answer the user's question?",
score_type="binary",
)
judge = client.agent_judges.update(
judge_id=judge.judge_id,
prompt="Updated rubric prompt.",
)Attributes
judge_id?str
Unique judge identifier on the Judgment platform.
strname?str
Human-readable name of the judge (unique per project).
strprompt?str
Rubric prompt template used by the Agent judge.
strmodel?str
Model the platform resolved from reasoning_level.
strscore_type?ScoreType
One of "numeric", "binary", or "categorical".
ScoreTypedescription?Optional[str]
Optional description stored on the judge version.
Optional[str]Nonejudge_description?Optional[str]
Optional human-readable description shown in the UI.
Optional[str]Nonecategories?Optional[List[Dict[str, Any]]]
Choice list for categorical judges (e.g. [{"name": "good", "description": "..."}, ...]).
Optional[List[Dict[str, Any]]]field(default=None)min_score?Optional[float]
Lower bound for numeric judges (defaults to 0).
Optional[float]Nonemax_score?Optional[float]
Upper bound for numeric judges (defaults to 1).
Optional[float]Nonemajor_version?Optional[int]
Major version this judge was read at.
Optional[int]Noneminor_version?Optional[int]
Minor version this judge was read at.
Optional[int]Nonereasoning_level?Optional[ReasoningLevel]
One of "lite", "normal", or "high".
Optional[ReasoningLevel]None