---
title: "Run your first test suite"
description: "Create a test suite with one case, run a sample agent against it, and read the result."
sidebar:
  label: "Run your first test suite"
seo:
  title: "Run your first test suite | Judgment Docs"
  description: "Create a test suite with one case, run an agent with the test attributes on its root span, and read the passed result."
---

In this tutorial, you create a judge and a test suite with one case, run a
sample agent against it, and read the result. The tutorial takes a few minutes
and uses one judge evaluation.

## Prerequisites

- The judgeval SDK, with the `JUDGMENT_API_KEY` and `JUDGMENT_ORG_ID`
  environment variables set.
- A project in Judgment. The script uses one named `support-agent`.

## Create the judge

In the `support-agent` project, open **Judges** and create a judge that asks
whether the final answer tells the customer they were refunded, with two
labels: `Yes` and `No`. See [Create a judge](/documentation/judges/create-a-judge).

Open the judge and note two things for the script:

- Its ID, the last part of the page URL (`…/judges/<judge_id>`).
- Its version from the version menu. For `v1.0`, the major version is `1` and
  the minor version is `0`.

## Run the script

```python
import time
from judgeval import Judgeval, Tracer

PROJECT = "support-agent"
JUDGE_ID = "<judge_id>"            # from the judge's page URL
JUDGE_MAJOR, JUDGE_MINOR = 1, 0    # from the judge's version menu

client = Judgeval(project_name=PROJECT)
Tracer.init(project_name=PROJECT)

# 1. A test suite with one case that expects the judge to return Yes.
suite = client.test_suites.create(name="my-first-test-suite")
client.test_suites.add_cases(suite["test_suite_id"], [{
    "input": {"question": "I was charged twice"},
    "expected": {
        JUDGE_ID: {
            "judge_major_version": JUDGE_MAJOR,
            "judge_minor_version": JUDGE_MINOR,
            "op": "eq",
            "expected": "Yes",
        },
    },
}])
client.test_suites.publish(suite["test_suite_id"])  # version 1 is now runnable

# 2. Start a run and run the agent once per case.
@Tracer.observe(span_type="agent")
def support_agent(question):
    return "Sorry about that! I've refunded the duplicate charge."

run = client.test_runs.start(suite["test_suite_id"])
for case in run["cases"]:
    with Tracer.test_case(run["run_id"], case["case_id"]):
        support_agent(case["input"]["question"])
Tracer.force_flush()

# 3. Wait until the case is scored, then print the result.
while client.test_runs.get(run["run_id"])["cases_scored"] < len(run["cases"]):
    time.sleep(5)
print(client.test_runs.results(run["run_id"]))
```

Test suite names must be unique in the project. To run the script again,
change the name or delete the earlier test suite with
`client.test_suites.delete(test_suite_id)`.

## Expected result

Within about a minute, the script prints a single passing case:

```json
{
  "results": [{
    "case_id": "…", "trace_id": "…", "scored_at": "…", "passed": true,
    "checks": [{
      "judge_id": "…", "judge_major_version": 1, "judge_minor_version": 0,
      "op": "eq", "expected": "Yes", "judge_output": "Yes", "passed": true,
      "reason": "The final answer tells the customer they were refunded. <citation …>",
      "error_message": null
    }]
  }]
}
```

The `reason` cites the trace spans the judge relied on with `<citation>` tags,
shortened here.

To see a failing case, change the sample agent's answer so that it no longer
mentions a refund, and start another run. The same case then fails with
`judge_output: "No"`.

## Next steps

- [Create and edit a test suite](/documentation/tests/create-a-test-suite) with your own cases.
- [Connect your real agent](/documentation/tests/connect-your-agent).