Dataset
A schema-enforced collection of Example objects on the Judgment platform.
Datasets are created and retrieved via client.datasets. Every
dataset has a JSON Schema that all examples are validated against
server-side. Once you have a Dataset, you can append examples, add
trace-backed examples, list versions, iterate over examples, export
to JSON/YAML, or display a rich table preview.
Create a dataset and add examples:
dataset = client.datasets.create(
name="golden-set",
schema={
"type": "object",
"properties": {
"input": {"type": "string"},
"expected_output": {"type": "string"},
},
},
)
dataset.add_examples([
Example.create(input="What is AI?", expected_output="Artificial Intelligence"),
])Retrieve and iterate:
dataset = client.datasets.get(name="golden-set")
for example in dataset:
print(example["input"])Attributes
Prop
Type
add_from_json()
Upload examples from a JSON file.
The file should contain a JSON array of objects, where each object
has the fields you want as example properties (e.g. input,
actual_output, expected_output).
dataset.add_from_json("./data/golden-set.json")def add_from_json(file_path, batch_size=100) -> None:Parameters
Prop
Type
Returns
None
add_from_yaml()
Upload examples from a YAML file.
Same as add_from_json but reads YAML format.
def add_from_yaml(file_path, batch_size=100) -> None:Parameters
Prop
Type
Returns
None
add_examples()
Append Example objects to this dataset.
Examples are validated server-side against the dataset schema and uploaded in batches with a progress bar. Each successful batch advances the dataset version. Accepts any iterable, including generators.
Raises
TypeError: If a single Example is passed instead of a list.
JudgmentValidationError: If examples fail schema validation.
def add_examples(examples, batch_size=100) -> None:Parameters
Prop
Type
Returns
None
versions()
List all versions of this dataset, newest first.
def versions() -> typing.List:Returns
typing.List - A list of DatasetVersion objects.
delete()
Delete this dataset from the platform.
Dependent test configs are deleted along with the dataset.
def delete() -> None:Returns
None
save_as()
Export the dataset to a local JSON or YAML file.
dataset = client.datasets.get(name="golden-set")
dataset.save_as("json", dir_path="./exports")def save_as(file_type, dir_path, save_name=None) -> None:Parameters
Prop
Type
Returns
None
display()
Print a formatted table preview to the terminal.
def display(max_examples=5) -> None:Parameters
Prop
Type
Returns
None