Example

A single evaluation example with flexible key-value properties.

Use Example.create() to build examples with any fields your scorers need. Common fields include input, actual_output, expected_output, and retrieval_context, but you can add any custom fields.

Access properties with bracket notation: example["input"].

Create an example for evaluating a Q&A system:

example = Example.create(
    input="What is the capital of France?",
    actual_output="Paris is the capital of France.",
    expected_output="Paris",
)

Include retrieval context for RAG evaluation:

example = Example.create(
    input="What is Python?",
    actual_output="A programming language.",
    retrieval_context=["Python is a high-level programming language..."],
)

Access properties:

print(example["input"])      # "What is Python?"
print("input" in example)    # True

Attributes

create()

Create an example with the given properties.

def create(**kwargs) -> Example:

Parameters

Returns

Example - A new Example instance.

to_dict()

Serialize the example to a dictionary suitable for the API.

def to_dict() -> APIExample:

Returns

APIExample