Example

Represents a single evaluation example containing input data and expected outputs for testing AI systems. In v1, examples use a flexible property-based system that allows you to store any fields you need.

example_id

:str

Unique identifier for the example. Auto-generated if not provided.

Example: "550e8400-e29b-41d4-a716-446655440000"

created_at

:str

ISO timestamp of when the example was created. Auto-generated if not provided.

Example: "2024-01-15T10:30:00.000000"

name

:Optional[str]

Optional name identifier for the example

Example: "geography_question_1"

trace

:Optional[Trace]

An optional Trace associated with the example, allowing examples to carry their execution trace.

Example: None

Methods

properties

:Dict[str, Any]

Property that returns a copy of all properties stored on the example.

getitem(key: str)

:Any

Get a property value using bracket notation (example["key"]). Raises KeyError if the property does not exist.

contains(key: str)

:bool

Check whether a property exists using the in operator ("key" in example).

to_dict()

:Dict[str, Any]

Convert the example to a dictionary format for API serialization.

Class Methods

Example.create(**kwargs)

:Example

Create a new example with properties set from keyword arguments. This is the recommended way to create examples.

Usage Examples

from judgeval.v1.data.example import Example

# Create example using the create() class method (recommended)
example = Example.create(
    input="What is 2 + 2?",
    expected_output="4"
)

# Create example with evaluation results
evaluated_example = Example.create(
    input="What is the capital of France?",
    expected_output="Paris",
    actual_output="Paris is the capital city of France.",
    metadata={
        "category": "geography",
        "difficulty": "easy"
    }
)

# RAG example with retrieval context
rag_example = Example.create(
    input="Explain quantum computing",
    expected_output="Quantum computing uses quantum mechanical phenomena...",
    actual_output="Quantum computing is a revolutionary technology...",
    retrieval_context="According to research papers: Quantum computing leverages quantum mechanics...",
    additional_metadata={
        "model_version": "gpt-5.2",
        "temperature": 0.7,
        "retrieval_score": 0.95
    }
)

# Access properties using bracket notation
example = Example.create(input="What is the capital of France?", expected_output="Paris")
input_value = example["input"]
all_properties = example.properties  # Returns a copy of all properties

# Check whether a property exists
if "expected_output" in example:
    print(example["expected_output"])

Common Properties

While the Example class uses a flexible property system, these are common property names used in evaluations:

input: The input prompt or query to be evaluated
expected_output: The expected or ideal response for comparison
actual_output: The actual response generated by the system
retrieval_context: Additional context retrieved from external sources (e.g., RAG systems)
metadata: Additional context or information about the example
additional_metadata: Extended metadata for storing custom fields and evaluation-specific information