Example

A single evaluation example with flexible key-value properties.

Use Example.create() to build examples with any fields your scorers need. Common fields include input, actual_output, expected_output, and retrieval_context, but you can add any custom fields.

Access properties with bracket notation: example["input"].

Create an example for evaluating a Q&A system:

example = Example.create(
    input="What is the capital of France?",
    actual_output="Paris is the capital of France.",
    expected_output="Paris",
)

Include retrieval context for RAG evaluation:

example = Example.create(
    input="What is Python?",
    actual_output="A programming language.",
    retrieval_context=["Python is a high-level programming language..."],
)

Access properties:

print(example["input"])      # "What is Python?"
print("input" in example)    # True

Attributes

PropType

example_id?str

Typestr

Defaultfield(default_factory=(lambda: str(uuid.uuid4())))

created_at?str

Typestr

Defaultfield(default_factory=(lambda: datetime.utcnow().isoformat()))

name?Optional[str]

TypeOptional[str]

DefaultNone

trace?Optional[Trace]

TypeOptional[Trace]

DefaultNone

properties?Dict[str, Any]

Return a copy of the example's custom properties.

TypeDict[str, Any]

create()

Create an example with the given properties.

def create(**kwargs) -> Example:

Parameters

PropType

kwargs?Any

TypeAny

Default{}

Returns

Example - A new Example instance.

to_dict()

Serialize the example to a dictionary suitable for the API.

def to_dict() -> APIExample:

Returns

APIExample