Example

Represents a single evaluation example containing input data and expected outputs for testing AI systems.

inputrequired:str

The input prompt or query to be evaluated

Example:

"What is the capital of France?"

expected_output:str

The expected or ideal response for comparison during evaluation

Example: "The capital of France is Paris."

actual_output:str

The actual response generated by the system being evaluated

Example: "Paris is the capital city of France."

retrieval_context:str

Additional context retrieved from external sources (e.g., RAG systems)

Example:

"According to Wikipedia: Paris is the capital and most populous city of France..."

additional_metadata:Dict[str, Any]

Extended metadata for storing custom fields and evaluation-specific information

Example:

{
    "model_version": "gpt-4-0125",
    "temperature": 0.7,
    "response_time_ms": 1250
}

metadata:Dict[str, Any]

Additional context or information about the example

Example:

{
    "category": "geography",
    "difficulty": "easy",
    "source": "world_facts_dataset"
}

Usage Examples

from judgeval.data import Example

# Basic example
example = Example(
    input="What is 2 + 2?",
    expected_output="4"
)

# Example with evaluation results
evaluated_example = Example(
    input="What is the capital of France?",
    expected_output="Paris",
    actual_output="Paris is the capital city of France.",
    metadata={
        "category": "geography",
        "difficulty": "easy"
    }
)

# RAG example with retrieval context
rag_example = Example(
    input="Explain quantum computing",
    expected_output="Quantum computing uses quantum mechanical phenomena...",
    actual_output="Quantum computing is a revolutionary technology...",
    retrieval_context="According to research papers: Quantum computing leverages quantum mechanics...",
    additional_metadata={
        "model_version": "gpt-4-0125",
        "temperature": 0.7,
        "retrieval_score": 0.95
    }
)

Example

Usage Examples

On this page