---
title: TestSuitesFactory
seo:
  title: TestSuitesFactory — Python SDK
  description: >-
    Create test suites and edit their cases. Access via client.test_suites.
    (Python SDK)
description: Create test suites and edit their cases. Access via client.test_suites.
---

Cases change only in a suite's draft (version + 1). New cases go straight
into the draft; to edit or remove published cases, call `create_draft`
first.
`publish` freezes the draft as the next version.

```python
suite = client.test_suites.create(name="support-agent-regressions")
client.test_suites.add_cases(
    suite["test_suite_id"],
    [
        {
            "input": {"prompt": "I want a refund"},
            "expected": {
                "<judge-id>": {
                    "judge_major_version": 1,
                    "judge_minor_version": 0,
                    "op": "eq",
                    "expected": "refund",
                }
            },
            "tags": ["billing"],
        }
    ],
)
client.test_suites.publish(suite["test_suite_id"])
```

## create()

Create an empty test suite.

```python
def create(name, description=None, labels=None) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `name` | `str` | - | |
| `description?` | `Optional[str]` | `None` | |
| `labels?` | `Optional[List[str]]` | `None` | |

### Returns

`Dict[str, Any]`

***

## list()

List the project's test suites, newest first (`test_suites` and `next_offset`).

```python
def list(offset=0, limit=50) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `offset?` | `int` | `0` | |
| `limit?` | `int` | `50` | |

### Returns

`Dict[str, Any]`

***

## delete()

Delete a test suite with all its versions, runs and results.

```python
def delete(test_suite_id) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |

### Returns

`Dict[str, Any]`

***

## list\_cases()

List the cases in a published version (default: latest) or the draft.

```python
def list_cases(test_suite_id, version=None, offset=0, limit=100) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |
| `version?` | `Optional[int \| Literal['draft']]` | `None` | |
| `offset?` | `int` | `0` | |
| `limit?` | `int` | `100` | |

### Returns

`Dict[str, Any]`

***

## add\_cases()

Add cases to the draft. Returns their `case_ids` in order.

Each case is `{"input": ..., "expected": {judge_id: check}, "tags": [...],
"name": ..., "metadata": {...}}`; only `input` is required. A check is
`{"judge_major_version", "judge_minor_version", "op", "expected"}`,
where `op` is one of eq, ne, lt, lte, gt, gte, in, not\_in.

```python
def add_cases(test_suite_id, cases) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |
| `cases` | `List[Dict[str, Any]]` | - | |

### Returns

`Dict[str, Any]`

***

## update\_case()

Replace a case in the draft. Call `create_draft` first for published cases.

```python
def update_case(test_suite_id, case_id, case) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |
| `case_id` | `str` | - | |
| `case` | `Dict[str, Any]` | - | |

### Returns

`Dict[str, Any]`

***

## remove\_case()

Remove a case from the draft. Call `create_draft` first for published cases.

```python
def remove_case(test_suite_id, case_id) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |
| `case_id` | `str` | - | |

### Returns

`Dict[str, Any]`

***

## create\_draft()

Copy the published cases into the draft so they can be edited.

```python
def create_draft(test_suite_id) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |

### Returns

`Dict[str, Any]`

***

## publish()

Freeze the draft as the next version. Returns the new `version`.

```python
def publish(test_suite_id) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |

### Returns

`Dict[str, Any]`

***

## discard()

Throw away the draft's changes.

```python
def discard(test_suite_id) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `test_suite_id` | `str` | - | |

### Returns

`Dict[str, Any]`