Example
A single evaluation example containing input data and expected outputs
Represents a single evaluation example containing input data and expected outputs for testing AI systems. In v1, examples use a flexible property-based system that allows you to store any fields you need.
example_id
:strUnique identifier for the example. Auto-generated if not provided.
"550e8400-e29b-41d4-a716-446655440000"
created_at
:strISO timestamp of when the example was created. Auto-generated if not provided.
"2024-01-15T10:30:00.000000"
name
:Optional[str]Optional name identifier for the example
"geography_question_1"
Methods
set_property(key: str, value: Any)
:ExampleSet a property on the example. Returns self for method chaining.
get_property(key: str)
:AnyGet a property value by key. Returns None if the property doesn't exist.
properties
:Dict[str, Any]Property that returns a copy of all properties stored on the example.
to_dict()
:Dict[str, Any]Convert the example to a dictionary format for API serialization.
Class Methods
Example.create(**kwargs)
:ExampleCreate a new example with properties set from keyword arguments. This is the recommended way to create examples.
Usage Examples
from judgeval.v1.data.example import Example
# Create example using the create() class method (recommended)
example = Example.create(
input="What is 2 + 2?",
expected_output="4"
)
# Create example with evaluation results
evaluated_example = Example.create(
input="What is the capital of France?",
expected_output="Paris",
actual_output="Paris is the capital city of France.",
metadata={
"category": "geography",
"difficulty": "easy"
}
)
# RAG example with retrieval context
rag_example = Example.create(
input="Explain quantum computing",
expected_output="Quantum computing uses quantum mechanical phenomena...",
actual_output="Quantum computing is a revolutionary technology...",
retrieval_context="According to research papers: Quantum computing leverages quantum mechanics...",
additional_metadata={
"model_version": "gpt-5.2",
"temperature": 0.7,
"retrieval_score": 0.95
}
)
# Create example and set properties manually
example = Example(name="my_example")
example.set_property("input", "What is the capital of France?")
example.set_property("expected_output", "Paris")
# Access properties
input_value = example.get_property("input")
all_properties = example.properties # Returns a copy of all properties
# Method chaining
example = Example.create(
input="Question"
).set_property("expected_output", "Answer").set_property("category", "test")Common Properties
While the Example class uses a flexible property system, these are common property names used in evaluations:
input: The input prompt or query to be evaluatedexpected_output: The expected or ideal response for comparisonactual_output: The actual response generated by the systemretrieval_context: Additional context retrieved from external sources (e.g., RAG systems)metadata: Additional context or information about the exampleadditional_metadata: Extended metadata for storing custom fields and evaluation-specific information