CategoricalResponse

Response for classification-style scorers.

Subclass this and define a categories class variable listing every allowed category. The value is validated against this list.

class SentimentResponse(CategoricalResponse):
    categories = [
        Category(value="positive", description="Positive sentiment"),
        Category(value="neutral", description="Neutral sentiment"),
        Category(value="negative", description="Negative sentiment"),
    ]

class SentimentJudge(Judge[SentimentResponse]):
    async def score(self, data: Example) -> SentimentResponse:
        return SentimentResponse(
            value="positive",
            reason="The output has an enthusiastic tone.",
        )

Attributes

validate_value_in_categories()

def validate_value_in_categories():