ExternalJudgeFactory
Create external judges and submit their results on the Judgment platform.
Access via client.externalJudges.
const judge = await client.externalJudges.create({
name: "human-thumbs-up",
scoreType: "binary",
});
await client.externalJudges.submitResult({
judgeId: judge.judgeId,
target: { type: "trace", traceId: "<trace_id>" },
value: true,
});create()
Create a new external judge and its first version.
const judge = await client.externalJudges.create({
name: "topic-classifier",
scoreType: "categorical",
outputs: [
{ name: "billing", description: "Billing questions" },
{ name: "support", description: "Support requests" },
],
});async function create(options: { name: string; scoreType: ScoreType; judgeDescription?: string; outputs?: { name: string; description: string; }[]; }): Promise<ExternalJudge | null>Parameters
Prop
Type
Returns
Promise<ExternalJudge | null> - The newly created ExternalJudge, or null if the project is unresolved.
submitResult()
Submits a customer score under any judge for a trace, session, or conversation.
Resubmitting for the same judge and target replaces its current customer score. To correct a value, submit again for that judge and target. Each submission, including a retry, writes a new result row.
Exactly one of judgeId or judgeName must be provided; the score
is attached to that judge's production version. value must match
the judge's score type (boolean for binary, number for
numeric, or one of the judge's configured outputs' names for
categorical).
await client.externalJudges.submitResult({
judgeName: "human-thumbs-up",
target: { type: "trace", traceId: "<trace_id>" },
value: true,
reason: "Reviewer approved the response.",
});async function submitResult(options: { target, value, judgeId?, judgeName?, reason? }): Promise<string | null>Parameters
Prop
Type
Returns
Promise<string | null> - The id of the persisted score result, or null if the
project is unresolved.