---
title: JudgmentApiClient
seo:
  title: JudgmentApiClient — Python SDK
  description: >-
    Client for the public /v1 API: one method per Judgment MCP tool. (Python
    SDK)
description: 'Client for the public /v1 API: one method per Judgment MCP tool.'
---

Reach it as `client.api` on a `Judgeval` client, which supplies the
credentials. Methods on project resources take the project's ID, which
`list_projects()` returns. Request bodies are dicts and responses are dicts.

```python
from judgeval import Judgeval

client = Judgeval(project_name="search-assistant")

projects = client.api.list_projects()["projects"]
project_id = next(
    p["project_id"]
    for p in projects
    if p["project_name"] == "search-assistant"
)

judges = client.api.list_judges(project_id)
result = client.api.sql_query(
    project_id,
    {"sql": "SELECT trace_id, status FROM telemetry.traces LIMIT 5"},
)
print(result["rows"])
```

## list\_organizations()

List organization memberships ([`GET /v1/organizations/memberships`](/documentation/reference/api/v1/get-v1-organizations-memberships)).

```python
client.api.list_organizations()
```

```python
def list_organizations() -> ListOrganizationsResponse:
```

### Returns

`ListOrganizationsResponse`

***

## create\_project()

Add new project ([`POST /v1/projects`](/documentation/reference/api/v1/post-v1-projects)).

```python
client.api.create_project({"project_name": ...})
```

```python
def create_project(payload) -> AddProjectResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `payload` | `AddProjectRequest` | - | |

### Returns

`AddProjectResponse`

***

## get\_artifact\_revision()

Get artifact revision ([`GET /v1/projects/{projectId}/artifact-revisions`](/documentation/reference/api/v1/get-v1-projects-by-project-id-artifact-revisions)).

```python
client.api.get_artifact_revision(project_id)
```

```python
def get_artifact_revision(project_id, *, revision_id=None, artifact_id=None, version=None) -> GetArtifactRevisionResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `revision_id?` | `str \| None` | `None` | |
| `artifact_id?` | `str \| None` | `None` | |
| `version?` | `int \| None` | `None` | |

### Returns

`GetArtifactRevisionResponse`

***

## create\_artifact()

Create artifact ([`POST /v1/projects/{projectId}/artifacts`](/documentation/reference/api/v1/post-v1-projects-by-project-id-artifacts)).

```python
client.api.create_artifact(project_id, {"scope": ..., "name": ..., "filename": ..., "source_code": ...})
```

```python
def create_artifact(project_id, payload) -> CreateArtifactResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateArtifactRequest` | - | |

### Returns

`CreateArtifactResponse`

***

## create\_supabase\_artifact\_revision()

Create artifact revision ([`POST /v1/projects/{projectId}/artifacts/{artifactId}/revisions`](/documentation/reference/api/v1/post-v1-projects-by-project-id-artifacts-by-artifact-id-revisions)).

```python
client.api.create_supabase_artifact_revision(project_id, artifact_id, {"version": ..., "filename": ..., "source_code": ...})
```

```python
def create_supabase_artifact_revision(project_id, artifact_id, payload) -> CreateSupabaseArtifactRevisionResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `artifact_id` | `str` | - | |
| `payload` | `CreateSupabaseArtifactRevisionRequest` | - | |

### Returns

`CreateSupabaseArtifactRevisionResponse`

***

## test\_optimistic\_eval\_candidate()

Test optimistic eval candidate ([`POST /v1/projects/{projectId}/artifacts/optimistic-eval-tests`](/documentation/reference/api/v1/post-v1-projects-by-project-id-artifacts-optimistic-eval-tests)).

```python
client.api.test_optimistic_eval_candidate(project_id, {"trace_id": ..., "source_code": ..., "judge_id": ..., "judge_version_id": ...})
```

```python
def test_optimistic_eval_candidate(project_id, payload) -> TestOptimisticEvalCandidateResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `TestOptimisticEvalCandidateRequest` | - | |

### Returns

`TestOptimisticEvalCandidateResponse`

***

## test\_preprocessing\_candidate()

Test preprocessing candidate ([`POST /v1/projects/{projectId}/artifacts/preprocessing-tests`](/documentation/reference/api/v1/post-v1-projects-by-project-id-artifacts-preprocessing-tests)).

```python
client.api.test_preprocessing_candidate(project_id, {"trace_id": ..., "source_code": ...})
```

```python
def test_preprocessing_candidate(project_id, payload) -> TestPreprocessingCandidateResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `TestPreprocessingCandidateRequest` | - | |

### Returns

`TestPreprocessingCandidateResponse`

***

## list\_mappers()

List attribute mappers ([`GET /v1/projects/{projectId}/attribute-mappers`](/documentation/reference/api/v1/get-v1-projects-by-project-id-attribute-mappers)).

```python
client.api.list_mappers(project_id)
```

```python
def list_mappers(project_id) -> ListMappersResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |

### Returns

`ListMappersResponse`

***

## create\_mapper()

Create attribute mapper ([`POST /v1/projects/{projectId}/attribute-mappers`](/documentation/reference/api/v1/post-v1-projects-by-project-id-attribute-mappers)).

```python
client.api.create_mapper(project_id, {"name": ..., "rules": ...})
```

```python
def create_mapper(project_id, payload) -> CreateMapperResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateMapperRequest` | - | |

### Returns

`CreateMapperResponse`

***

## get\_mapper()

Get attribute mapper ([`GET /v1/projects/{projectId}/attribute-mappers/{configId}`](/documentation/reference/api/v1/get-v1-projects-by-project-id-attribute-mappers-by-config-id)).

```python
client.api.get_mapper(project_id, config_id)
```

```python
def get_mapper(project_id, config_id) -> GetMapperResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `config_id` | `str` | - | |

### Returns

`GetMapperResponse`

***

## update\_mapper()

Update attribute mapper ([`PATCH /v1/projects/{projectId}/attribute-mappers/{configId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-attribute-mappers-by-config-id)).

```python
client.api.update_mapper(project_id, config_id, {})
```

```python
def update_mapper(project_id, config_id, payload) -> UpdateMapperResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `config_id` | `str` | - | |
| `payload` | `UpdateMapperRequest` | - | |

### Returns

`UpdateMapperResponse`

***

## delete\_mapper()

Delete attribute mapper ([`DELETE /v1/projects/{projectId}/attribute-mappers/{configId}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-attribute-mappers-by-config-id)).

```python
client.api.delete_mapper(project_id, config_id)
```

```python
def delete_mapper(project_id, config_id) -> DeleteMapperResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `config_id` | `str` | - | |

### Returns

`DeleteMapperResponse`

***

## update\_judge\_label()

Update judge label ([`PATCH /v1/projects/{projectId}/behaviors/{behaviorId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-behaviors-by-behavior-id)).

```python
client.api.update_judge_label(project_id, behavior_id, {})
```

```python
def update_judge_label(project_id, behavior_id, payload) -> UpdateJudgeLabelResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `behavior_id` | `str` | - | |
| `payload` | `UpdateJudgeLabelRequest` | - | |

### Returns

`UpdateJudgeLabelResponse`

***

## get\_judge\_label()

Get judge label ([`GET /v1/projects/{projectId}/behaviors/{behaviorId}/analytics`](/documentation/reference/api/v1/get-v1-projects-by-project-id-behaviors-by-behavior-id-analytics)).

```python
client.api.get_judge_label(project_id, behavior_id)
```

```python
def get_judge_label(project_id, behavior_id, *, start_date=None, end_date=None) -> GetJudgeLabelResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `behavior_id` | `str` | - | |
| `start_date?` | `str \| None` | `None` | |
| `end_date?` | `str \| None` | `None` | |

### Returns

`GetJudgeLabelResponse`

***

## get\_conversation\_display\_rules()

Get conversation display rules ([`GET /v1/projects/{projectId}/conversation-display-rules`](/documentation/reference/api/v1/get-v1-projects-by-project-id-conversation-display-rules)).

```python
client.api.get_conversation_display_rules(project_id)
```

```python
def get_conversation_display_rules(project_id) -> GetConversationDisplayRulesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |

### Returns

`GetConversationDisplayRulesResponse`

***

## set\_conversation\_display\_rules()

Replace conversation display rules ([`PUT /v1/projects/{projectId}/conversation-display-rules`](/documentation/reference/api/v1/put-v1-projects-by-project-id-conversation-display-rules)).

```python
client.api.set_conversation_display_rules(project_id, {"rules": ...})
```

```python
def set_conversation_display_rules(project_id, payload) -> SetConversationDisplayRulesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `SetConversationDisplayRulesRequest` | - | |

### Returns

`SetConversationDisplayRulesResponse`

***

## create\_dataset()

Create offline-test dataset ([`POST /v1/projects/{projectId}/datasets`](/documentation/reference/api/v1/post-v1-projects-by-project-id-datasets)).

```python
client.api.create_dataset(project_id, {"name": ..., "schema": ...})
```

```python
def create_dataset(project_id, payload) -> CreateOfflineDatasetResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateOfflineDatasetRequest` | - | |

### Returns

`CreateOfflineDatasetResponse`

***

## list\_datasets()

Fetch offline-test datasets ([`GET /v1/projects/{projectId}/datasets`](/documentation/reference/api/v1/get-v1-projects-by-project-id-datasets)).

```python
client.api.list_datasets(project_id)
```

```python
def list_datasets(project_id) -> PullAllOfflineDatasetsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |

### Returns

`PullAllOfflineDatasetsResponse`

***

## delete\_dataset()

Delete offline-test dataset ([`DELETE /v1/projects/{projectId}/datasets/{datasetIdentifier}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-datasets-by-dataset-identifier)).

```python
client.api.delete_dataset(project_id, dataset_identifier)
```

```python
def delete_dataset(project_id, dataset_identifier) -> DeleteOfflineDatasetResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `dataset_identifier` | `str` | - | |

### Returns

`DeleteOfflineDatasetResponse`

***

## add\_examples\_to\_dataset()

Insert examples to offline-test dataset ([`POST /v1/projects/{projectId}/datasets/{datasetIdentifier}/examples`](/documentation/reference/api/v1/post-v1-projects-by-project-id-datasets-by-dataset-identifier-examples)).

```python
client.api.add_examples_to_dataset(project_id, dataset_identifier, {"examples": ...})
```

```python
def add_examples_to_dataset(project_id, dataset_identifier, payload) -> AddDatasetItemsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `dataset_identifier` | `str` | - | |
| `payload` | `InsertOfflineDatasetExamplesRequest` | - | |

### Returns

`AddDatasetItemsResponse`

***

## remove\_examples\_from\_dataset()

Remove examples from offline-test dataset ([`POST /v1/projects/{projectId}/datasets/{datasetIdentifier}/examples/remove`](/documentation/reference/api/v1/post-v1-projects-by-project-id-datasets-by-dataset-identifier-examples-remove)).

```python
client.api.remove_examples_from_dataset(project_id, dataset_identifier, {"example_ids": ...})
```

```python
def remove_examples_from_dataset(project_id, dataset_identifier, payload) -> RemoveExamplesFromDatasetResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `dataset_identifier` | `str` | - | |
| `payload` | `RemoveExamplesFromDatasetRequest` | - | |

### Returns

`RemoveExamplesFromDatasetResponse`

***

## get\_dataset()

Fetch dataset page ([`GET /v1/projects/{projectId}/datasets/{datasetIdentifier}/page`](/documentation/reference/api/v1/get-v1-projects-by-project-id-datasets-by-dataset-identifier-page)).

```python
client.api.get_dataset(project_id, dataset_identifier)
```

```python
def get_dataset(project_id, dataset_identifier, *, version=None, limit=None, cursor_created_at=None, cursor_example_id=None) -> OfflineDatasetPageResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `dataset_identifier` | `str` | - | |
| `version?` | `float \| None` | `None` | |
| `limit?` | `float \| None` | `None` | |
| `cursor_created_at?` | `str \| None` | `None` | |
| `cursor_example_id?` | `str \| None` | `None` | |

### Returns

`OfflineDatasetPageResponse`

***

## list\_issues()

List issues ([`GET /v1/projects/{projectId}/issues`](/documentation/reference/api/v1/get-v1-projects-by-project-id-issues)).

```python
client.api.list_issues(project_id)
```

```python
def list_issues(project_id, *, user_status=None, production_activity=None, priority=None) -> ListIssuesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `user_status?` | `str \| None` | `None` | |
| `production_activity?` | `str \| None` | `None` | |
| `priority?` | `str \| None` | `None` | |

### Returns

`ListIssuesResponse`

***

## create\_issue()

Create issue ([`POST /v1/projects/{projectId}/issues`](/documentation/reference/api/v1/post-v1-projects-by-project-id-issues)).

```python
client.api.create_issue(project_id, {"title": ..., "diagnosis": ..., "cited_items": ...})
```

```python
def create_issue(project_id, payload) -> CreateIssueResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateIssueRequest` | - | |

### Returns

`CreateIssueResponse`

***

## get\_issue()

Get issue ([`GET /v1/projects/{projectId}/issues/{issueId}`](/documentation/reference/api/v1/get-v1-projects-by-project-id-issues-by-issue-id)).

```python
client.api.get_issue(project_id, issue_id)
```

```python
def get_issue(project_id, issue_id) -> GetIssueResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `issue_id` | `str` | - | |

### Returns

`GetIssueResponse`

***

## update\_issue()

Update issue ([`PATCH /v1/projects/{projectId}/issues/{issueId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-issues-by-issue-id)).

```python
client.api.update_issue(project_id, issue_id, {})
```

```python
def update_issue(project_id, issue_id, payload) -> UpdateIssueResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `issue_id` | `str` | - | |
| `payload` | `UpdateIssueRequest` | - | |

### Returns

`UpdateIssueResponse`

***

## list\_judges()

List judges ([`GET /v1/projects/{projectId}/judges`](/documentation/reference/api/v1/get-v1-projects-by-project-id-judges)).

```python
client.api.list_judges(project_id)
```

```python
def list_judges(project_id, *, version=None) -> ListJudgesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `version?` | `str \| None` | `None` | |

### Returns

`ListJudgesResponse`

***

## create\_judge()

Create judge ([`POST /v1/projects/{projectId}/judges`](/documentation/reference/api/v1/post-v1-projects-by-project-id-judges)).

```python
client.api.create_judge(project_id, {"judge": ...})
```

```python
def create_judge(project_id, payload) -> CreateJudgeResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateJudgeRequest` | - | |

### Returns

`CreateJudgeResponse`

***

## update\_judge()

Update judge ([`PATCH /v1/projects/{projectId}/judges/{judgeId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-judges-by-judge-id)).

```python
client.api.update_judge(project_id, judge_id, {"update": ...})
```

```python
def update_judge(project_id, judge_id, payload) -> UpdateJudgeResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |
| `payload` | `UpdateJudgeRequest` | - | |

### Returns

`UpdateJudgeResponse`

***

## delete\_judge()

Delete judge ([`DELETE /v1/projects/{projectId}/judges/{judgeId}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-judges-by-judge-id)).

```python
client.api.delete_judge(project_id, judge_id)
```

```python
def delete_judge(project_id, judge_id) -> DeleteJudgeResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |

### Returns

`DeleteJudgeResponse`

***

## set\_judge\_action\_groups()

Set judge action groups ([`POST /v1/projects/{projectId}/judges/{judgeId}/action-groups`](/documentation/reference/api/v1/post-v1-projects-by-project-id-judges-by-judge-id-action-groups)).

```python
client.api.set_judge_action_groups(project_id, judge_id, {"source_version": ..., "target_version": ..., "promote_to_production": ..., "action_groups": ...})
```

```python
def set_judge_action_groups(project_id, judge_id, payload) -> SetJudgeActionGroupsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |
| `payload` | `SetJudgeActionGroupsRequest` | - | |

### Returns

`SetJudgeActionGroupsResponse`

***

## get\_judge()

Get judge ([`POST /v1/projects/{projectId}/judges/{judgeId}/lookup`](/documentation/reference/api/v1/post-v1-projects-by-project-id-judges-by-judge-id-lookup)).

```python
client.api.get_judge(project_id, judge_id, {})
```

```python
def get_judge(project_id, judge_id, payload) -> GetJudgeResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |
| `payload` | `GetJudgeRequest` | - | |

### Returns

`GetJudgeResponse`

***

## set\_judge\_optimistic\_eval\_enabled()

Set judge version optimistic eval ([`PUT /v1/projects/{projectId}/judges/{judgeId}/versions/{judgeVersionId}/optimistic-eval`](/documentation/reference/api/v1/put-v1-projects-by-project-id-judges-by-judge-id-versions-by-judge-version-id-optimistic-eval)).

```python
client.api.set_judge_optimistic_eval_enabled(project_id, judge_id, judge_version_id, {"enabled": ...})
```

```python
def set_judge_optimistic_eval_enabled(project_id, judge_id, judge_version_id, payload) -> SetJudgeOptimisticEvalEnabledResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |
| `judge_version_id` | `str` | - | |
| `payload` | `SetJudgeOptimisticEvalEnabledRequest` | - | |

### Returns

`SetJudgeOptimisticEvalEnabledResponse`

***

## set\_judge\_preprocessor\_artifact\_revision()

Set judge version preprocessing artifact ([`PUT /v1/projects/{projectId}/judges/{judgeId}/versions/{judgeVersionId}/preprocessor-artifact-revision`](/documentation/reference/api/v1/put-v1-projects-by-project-id-judges-by-judge-id-versions-by-judge-version-id-preprocessor-artifact-revision)).

```python
client.api.set_judge_preprocessor_artifact_revision(project_id, judge_id, judge_version_id, {"preprocessor_artifact_revision_id": ...})
```

```python
def set_judge_preprocessor_artifact_revision(project_id, judge_id, judge_version_id, payload) -> SetJudgePreprocessorArtifactRevisionResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `judge_id` | `str` | - | |
| `judge_version_id` | `str` | - | |
| `payload` | `SetJudgePreprocessorArtifactRevisionRequest` | - | |

### Returns

`SetJudgePreprocessorArtifactRevisionResponse`

***

## list\_judge\_action\_groups()

List judge action groups ([`GET /v1/projects/{projectId}/judges/action-groups`](/documentation/reference/api/v1/get-v1-projects-by-project-id-judges-action-groups)).

```python
client.api.list_judge_action_groups(project_id)
```

```python
def list_judge_action_groups(project_id, *, version=None) -> ListJudgeActionGroupsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `version?` | `str \| None` | `None` | |

### Returns

`ListJudgeActionGroupsResponse`

***

## list\_memory\_entries()

List memory entries ([`GET /v1/projects/{projectId}/memory/entries`](/documentation/reference/api/v1/get-v1-projects-by-project-id-memory-entries)).

```python
client.api.list_memory_entries(project_id)
```

```python
def list_memory_entries(project_id) -> ListMemoryEntriesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |

### Returns

`ListMemoryEntriesResponse`

***

## write\_memory\_file()

Write a memory file ([`PUT /v1/projects/{projectId}/memory/files`](/documentation/reference/api/v1/put-v1-projects-by-project-id-memory-files)).

```python
client.api.write_memory_file(project_id, {"name": ..., "body": ...})
```

```python
def write_memory_file(project_id, payload) -> WriteMemoryFileResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `WriteMemoryFileRequest` | - | |

### Returns

`WriteMemoryFileResponse`

***

## read\_memory\_files()

Fetch memory files ([`POST /v1/projects/{projectId}/memory/files/fetch`](/documentation/reference/api/v1/post-v1-projects-by-project-id-memory-files-fetch)).

```python
client.api.read_memory_files(project_id, {})
```

```python
def read_memory_files(project_id, payload) -> ReadMemoryFilesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `ReadMemoryFilesRequest` | - | |

### Returns

`ReadMemoryFilesResponse`

***

## search\_memory\_files()

Search memory files ([`POST /v1/projects/{projectId}/memory/search`](/documentation/reference/api/v1/post-v1-projects-by-project-id-memory-search)).

```python
client.api.search_memory_files(project_id, {"query": ...})
```

```python
def search_memory_files(project_id, payload) -> SearchMemoryFilesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `SearchMemoryFilesRequest` | - | |

### Returns

`SearchMemoryFilesResponse`

***

## list\_scheduled\_tasks()

List scheduled tasks ([`GET /v1/projects/{projectId}/scheduled-tasks`](/documentation/reference/api/v1/get-v1-projects-by-project-id-scheduled-tasks)).

```python
client.api.list_scheduled_tasks(project_id)
```

```python
def list_scheduled_tasks(project_id) -> ListScheduledTasksResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |

### Returns

`ListScheduledTasksResponse`

***

## create\_scheduled\_task()

Create a scheduled task ([`POST /v1/projects/{projectId}/scheduled-tasks`](/documentation/reference/api/v1/post-v1-projects-by-project-id-scheduled-tasks)).

```python
client.api.create_scheduled_task(project_id, {"name": ...})
```

```python
def create_scheduled_task(project_id, payload) -> CreateScheduledTaskResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateScheduledTaskRequest` | - | |

### Returns

`CreateScheduledTaskResponse`

***

## get\_scheduled\_task()

Get a scheduled task ([`GET /v1/projects/{projectId}/scheduled-tasks/{taskId}`](/documentation/reference/api/v1/get-v1-projects-by-project-id-scheduled-tasks-by-task-id)).

```python
client.api.get_scheduled_task(project_id, task_id)
```

```python
def get_scheduled_task(project_id, task_id) -> GetScheduledTaskResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `task_id` | `str` | - | |

### Returns

`GetScheduledTaskResponse`

***

## update\_scheduled\_task()

Update a scheduled task ([`PATCH /v1/projects/{projectId}/scheduled-tasks/{taskId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-scheduled-tasks-by-task-id)).

```python
client.api.update_scheduled_task(project_id, task_id, {})
```

```python
def update_scheduled_task(project_id, task_id, payload) -> UpdateScheduledTaskResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `task_id` | `str` | - | |
| `payload` | `UpdateScheduledTaskRequest` | - | |

### Returns

`UpdateScheduledTaskResponse`

***

## sql\_query()

Run a tenant-scoped virtual SQL query ([`POST /v1/projects/{projectId}/sql`](/documentation/reference/api/v1/post-v1-projects-by-project-id-sql)).

```python
client.api.sql_query(project_id, {"sql": ...})
```

```python
def sql_query(project_id, payload) -> PublicSqlResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `PublicSqlRequest` | - | |

### Returns

`PublicSqlResponse`

***

## list\_test\_configs()

Fetch test configs ([`GET /v1/projects/{projectId}/test-configs`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-configs)).

```python
client.api.list_test_configs(project_id)
```

```python
def list_test_configs(project_id, *, dataset_id=None) -> TestConfigsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `dataset_id?` | `str \| None` | `None` | |

### Returns

`TestConfigsResponse`

***

## create\_test\_config()

Create test config ([`POST /v1/projects/{projectId}/test-configs`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-configs)).

```python
client.api.create_test_config(project_id, {"name": ...})
```

```python
def create_test_config(project_id, payload) -> TestConfigResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateTestConfigRequest` | - | |

### Returns

`TestConfigResponse`

***

## update\_test\_config()

Update test config ([`PATCH /v1/projects/{projectId}/test-configs/{testConfigId}`](/documentation/reference/api/v1/patch-v1-projects-by-project-id-test-configs-by-test-config-id)).

```python
client.api.update_test_config(project_id, test_config_id, {})
```

```python
def update_test_config(project_id, test_config_id, payload) -> TestConfigResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_config_id` | `str` | - | |
| `payload` | `UpdateTestConfigRequest` | - | |

### Returns

`TestConfigResponse`

***

## delete\_test\_config()

Delete test config ([`DELETE /v1/projects/{projectId}/test-configs/{testConfigId}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-test-configs-by-test-config-id)).

```python
client.api.delete_test_config(project_id, test_config_id)
```

```python
def delete_test_config(project_id, test_config_id) -> DeleteTestConfigResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_config_id` | `str` | - | |

### Returns

`DeleteTestConfigResponse`

***

## list\_test\_runs()

Fetch test runs ([`GET /v1/projects/{projectId}/test-runs`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-runs)).

```python
client.api.list_test_runs(project_id)
```

```python
def list_test_runs(project_id, *, test_config_id=None, dataset_id=None, status=None, limit=None) -> TestRunsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_config_id?` | `str \| None` | `None` | |
| `dataset_id?` | `str \| None` | `None` | |
| `status?` | `str \| None` | `None` | |
| `limit?` | `float \| None` | `None` | |

### Returns

`TestRunsResponse`

***

## run\_test()

Prepare and queue test run ([`POST /v1/projects/{projectId}/test-runs`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-runs)).

```python
client.api.run_test(project_id, {"test_config_id": ...})
```

```python
def run_test(project_id, payload) -> PreparedTestRunResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateTestRunRequest` | - | |

### Returns

`PreparedTestRunResponse`

***

## get\_test\_run()

Fetch test run ([`GET /v1/projects/{projectId}/test-runs/{testRunId}`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-runs-by-test-run-id)).

```python
client.api.get_test_run(project_id, test_run_id)
```

```python
def get_test_run(project_id, test_run_id) -> TestRunResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_run_id` | `str` | - | |

### Returns

`TestRunResponse`

***

## cancel\_test\_run()

Cancel test run ([`POST /v1/projects/{projectId}/test-runs/{testRunId}/cancel`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-runs-by-test-run-id-cancel)).

```python
client.api.cancel_test_run(project_id, test_run_id)
```

```python
def cancel_test_run(project_id, test_run_id) -> CancelTestRunResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_run_id` | `str` | - | |

### Returns

`CancelTestRunResponse`

***

## get\_test\_run\_items()

Fetch test run items ([`GET /v1/projects/{projectId}/test-runs/{testRunId}/items`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-runs-by-test-run-id-items)).

```python
client.api.get_test_run_items(project_id, test_run_id)
```

```python
def get_test_run_items(project_id, test_run_id, *, limit=None, cursor=None) -> TestRunItemsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_run_id` | `str` | - | |
| `limit?` | `float \| None` | `None` | |
| `cursor?` | `str \| None` | `None` | |

### Returns

`TestRunItemsResponse`

***

## create\_test\_suite()

Create a test suite ([`POST /v1/projects/{projectId}/test-suites`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites)).

```python
client.api.create_test_suite(project_id, {"name": ...})
```

```python
def create_test_suite(project_id, payload) -> TestSuite:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `CreateTestSuiteRequest` | - | |

### Returns

`TestSuite`

***

## list\_test\_suites()

List test suites ([`GET /v1/projects/{projectId}/test-suites`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-suites)).

```python
client.api.list_test_suites(project_id)
```

```python
def list_test_suites(project_id, *, offset=None, limit=None) -> ListTestSuitesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `offset?` | `int \| None` | `None` | |
| `limit?` | `int \| None` | `None` | |

### Returns

`ListTestSuitesResponse`

***

## delete\_test\_suite()

Delete a test suite with its runs ([`DELETE /v1/projects/{projectId}/test-suites/{testSuiteId}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-test-suites-by-test-suite-id)).

```python
client.api.delete_test_suite(project_id, test_suite_id)
```

```python
def delete_test_suite(project_id, test_suite_id) -> DeleteTestSuiteResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |

### Returns

`DeleteTestSuiteResponse`

***

## list\_test\_cases()

List test cases in a version or the draft ([`GET /v1/projects/{projectId}/test-suites/{testSuiteId}/cases`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-suites-by-test-suite-id-cases)).

```python
client.api.list_test_cases(project_id, test_suite_id)
```

```python
def list_test_cases(project_id, test_suite_id, *, version=None, offset=None, limit=None) -> ListTestCasesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `version?` | `str \| None` | `None` | |
| `offset?` | `int \| None` | `None` | |
| `limit?` | `int \| None` | `None` | |

### Returns

`ListTestCasesResponse`

***

## add\_test\_cases()

Add cases to a test suite's draft ([`POST /v1/projects/{projectId}/test-suites/{testSuiteId}/cases`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-by-test-suite-id-cases)).

```python
client.api.add_test_cases(project_id, test_suite_id, {"cases": ...})
```

```python
def add_test_cases(project_id, test_suite_id, payload) -> AddTestCasesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `payload` | `AddTestCasesRequest` | - | |

### Returns

`AddTestCasesResponse`

***

## update\_test\_case()

Replace a case in the draft ([`PUT /v1/projects/{projectId}/test-suites/{testSuiteId}/cases/{caseId}`](/documentation/reference/api/v1/put-v1-projects-by-project-id-test-suites-by-test-suite-id-cases-by-case-id)).

```python
client.api.update_test_case(project_id, test_suite_id, case_id, {"input": ...})
```

```python
def update_test_case(project_id, test_suite_id, case_id, payload) -> UpdateTestCaseResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `case_id` | `str` | - | |
| `payload` | `TestCaseInput` | - | |

### Returns

`UpdateTestCaseResponse`

***

## remove\_test\_case()

Remove a case from the draft ([`DELETE /v1/projects/{projectId}/test-suites/{testSuiteId}/cases/{caseId}`](/documentation/reference/api/v1/delete-v1-projects-by-project-id-test-suites-by-test-suite-id-cases-by-case-id)).

```python
client.api.remove_test_case(project_id, test_suite_id, case_id)
```

```python
def remove_test_case(project_id, test_suite_id, case_id) -> RemoveTestCaseResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `case_id` | `str` | - | |

### Returns

`RemoveTestCaseResponse`

***

## discard\_test\_suite\_draft()

Throw away the draft's changes ([`POST /v1/projects/{projectId}/test-suites/{testSuiteId}/discard`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-by-test-suite-id-discard)).

```python
client.api.discard_test_suite_draft(project_id, test_suite_id)
```

```python
def discard_test_suite_draft(project_id, test_suite_id) -> DiscardTestSuiteDraftResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |

### Returns

`DiscardTestSuiteDraftResponse`

***

## create\_test\_suite\_draft()

Copy the published cases into the draft to edit ([`POST /v1/projects/{projectId}/test-suites/{testSuiteId}/draft`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-by-test-suite-id-draft)).

```python
client.api.create_test_suite_draft(project_id, test_suite_id)
```

```python
def create_test_suite_draft(project_id, test_suite_id) -> CreateTestSuiteDraftResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |

### Returns

`CreateTestSuiteDraftResponse`

***

## publish\_test\_suite()

Publish the draft as the next version ([`POST /v1/projects/{projectId}/test-suites/{testSuiteId}/publish`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-by-test-suite-id-publish)).

```python
client.api.publish_test_suite(project_id, test_suite_id)
```

```python
def publish_test_suite(project_id, test_suite_id) -> PublishTestSuiteResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |

### Returns

`PublishTestSuiteResponse`

***

## start\_test\_suite\_run()

Start a test run ([`POST /v1/projects/{projectId}/test-suites/{testSuiteId}/runs`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-by-test-suite-id-runs)).

```python
client.api.start_test_suite_run(project_id, test_suite_id, {})
```

```python
def start_test_suite_run(project_id, test_suite_id, payload) -> StartTestRunResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `payload` | `StartTestRunRequest` | - | |

### Returns

`StartTestRunResponse`

***

## list\_test\_suite\_runs()

List test runs ([`GET /v1/projects/{projectId}/test-suites/{testSuiteId}/runs`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-suites-by-test-suite-id-runs)).

```python
client.api.list_test_suite_runs(project_id, test_suite_id)
```

```python
def list_test_suite_runs(project_id, test_suite_id, *, offset=None, limit=None) -> ListTestRunsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `test_suite_id` | `str` | - | |
| `offset?` | `int \| None` | `None` | |
| `limit?` | `int \| None` | `None` | |

### Returns

`ListTestRunsResponse`

***

## get\_test\_suite\_run\_results()

Get test run results ([`GET /v1/projects/{projectId}/test-suites/runs/{runId}/results`](/documentation/reference/api/v1/get-v1-projects-by-project-id-test-suites-runs-by-run-id-results)).

```python
client.api.get_test_suite_run_results(project_id, run_id)
```

```python
def get_test_suite_run_results(project_id, run_id) -> TestResultsResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `run_id` | `str` | - | |

### Returns

`TestResultsResponse`

***

## score\_test\_run()

Score a test run ([`POST /v1/projects/{projectId}/test-suites/runs/{runId}/score`](/documentation/reference/api/v1/post-v1-projects-by-project-id-test-suites-runs-by-run-id-score)).

```python
client.api.score_test_run(project_id, run_id)
```

```python
def score_test_run(project_id, run_id) -> ScoreTestRunResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `run_id` | `str` | - | |

### Returns

`ScoreTestRunResponse`

***

## list\_trace\_health\_failures()

List traces failing a trace health check ([`GET /v1/projects/{projectId}/trace-health/checks/{checkId}/traces`](/documentation/reference/api/v1/get-v1-projects-by-project-id-trace-health-checks-by-check-id-traces)).

```python
client.api.list_trace_health_failures(project_id, check_id, start_time=..., end_time=...)
```

```python
def list_trace_health_failures(project_id, check_id, *, start_time, end_time, category=None, limit=None, cursor_trace_started_at=None, cursor_trace_id=None) -> ListTraceHealthFailuresResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `check_id` | `str` | - | |
| `start_time` | `str` | - | |
| `end_time` | `str` | - | |
| `category?` | `str \| None` | `None` | |
| `limit?` | `int \| None` | `None` | |
| `cursor_trace_started_at?` | `str \| None` | `None` | |
| `cursor_trace_id?` | `str \| None` | `None` | |

### Returns

`ListTraceHealthFailuresResponse`

***

## get\_trace\_health\_summary()

Get trace health summary ([`GET /v1/projects/{projectId}/trace-health/summary`](/documentation/reference/api/v1/get-v1-projects-by-project-id-trace-health-summary)).

```python
client.api.get_trace_health_summary(project_id, start_time=..., end_time=...)
```

```python
def get_trace_health_summary(project_id, *, start_time, end_time, category=None) -> GetTraceHealthSummaryResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `start_time` | `str` | - | |
| `end_time` | `str` | - | |
| `category?` | `str \| None` | `None` | |

### Returns

`GetTraceHealthSummaryResponse`

***

## get\_trace\_health()

Get trace health for one trace ([`GET /v1/projects/{projectId}/trace-health/traces/{traceId}`](/documentation/reference/api/v1/get-v1-projects-by-project-id-trace-health-traces-by-trace-id)).

```python
client.api.get_trace_health(project_id, trace_id)
```

```python
def get_trace_health(project_id, trace_id, *, category=None) -> GetTraceHealthResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `trace_id` | `str` | - | |
| `category?` | `str \| None` | `None` | |

### Returns

`GetTraceHealthResponse`

***

## evaluate\_traces()

Evaluate stored traces ([`POST /v1/projects/{projectId}/traces/evaluate`](/documentation/reference/api/v1/post-v1-projects-by-project-id-traces-evaluate)).

```python
client.api.evaluate_traces(project_id, {"trace_ids": ...})
```

```python
def evaluate_traces(project_id, payload) -> EvaluateTracesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `EvaluateTracesRequest` | - | |

### Returns

`EvaluateTracesResponse`

***

## evaluate\_all\_traces()

Evaluate recent traces ([`POST /v1/projects/{projectId}/traces/evaluate-all`](/documentation/reference/api/v1/post-v1-projects-by-project-id-traces-evaluate-all)).

```python
client.api.evaluate_all_traces(project_id, {})
```

```python
def evaluate_all_traces(project_id, payload) -> EvaluateAllTracesResponse:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `project_id` | `str` | - | |
| `payload` | `EvaluateAllTracesRequest` | - | |

### Returns

`EvaluateAllTracesResponse`

***

## list\_projects()

List project summaries ([`GET /v1/projects/summaries`](/documentation/reference/api/v1/get-v1-projects-summaries)).

```python
client.api.list_projects()
```

```python
def list_projects() -> ListProjectsResponse:
```

### Returns

`ListProjectsResponse`

***

## discover\_schema()

Discover the virtual SQL schema ([`GET /v1/sql/schema`](/documentation/reference/api/v1/get-v1-sql-schema)).

```python
client.api.discover_schema()
```

```python
def discover_schema() -> PublicSqlSchemaResponse:
```

### Returns

`PublicSqlSchemaResponse`