---
title: "judgment automations"
description: "Manage automations (rules) that fire actions when metrics match conditions."
sidebar:
  label: "Automations"
seo:
  title: "judgment automations CLI Reference | Judgment Docs"
  description: "Command reference for `judgment automations`, used to manage automations that act on metrics in Judgment CLI workflows."
---

> **Warning**
>
> **The Judgment CLI is deprecated and no longer under active development.**
> This reference is retained for existing users. See the [CLI deprecation
> notice](/documentation/cli) for supported alternatives.


Manage automations (rules) that fire actions when metrics match conditions.

## Commands

| Command | Description |
| ------- | ----------- |
| [`automations create`](#automations-create) | Create an automation. |
| [`automations delete`](#automations-delete) | Delete an automation. |
| [`automations get`](#automations-get) | Get an automation by ID. |
| [`automations list`](#automations-list) | List automations. |
| [`automations update`](#automations-update) | Update an automation. |

### `automations create`

Create an automation.

Create an automation (rule) in a project. An automation watches behavior/latency/cost metrics and fires actions when its conditions match. Requires the developer role.

```bash
judgment automations create [OPTIONS] [[[ORG_ID] PROJECT_ID] NAME...]
```

**Arguments**

| Name | Required |
| ---- | -------- |
| `[[ORG_ID] PROJECT_ID] NAME` | no |

**Options**

| Flag | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
| <code style={{whiteSpace:"nowrap"}}>--organization-id</code>, <code style={{whiteSpace:"nowrap"}}>--org-id</code> | text | no | Organization ID. Defaults to JUDGMENT_ORG_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--organization</code>, <code style={{whiteSpace:"nowrap"}}>--org</code> | text | no | Organization name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--project-id</code> | text | no | Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--project</code> | text | no | Project name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--description</code> | text | no | Human-readable description shown in the UI. |
| <code style={{whiteSpace:"nowrap"}}>--conditions</code> | text | yes | JSON array of rule conditions. Each condition references a named metric/scorer on the project and a comparison. Items are ANDed or ORed together based on `combine_type` (`all` vs `any`).  **Condition shape:**   **Common scorer_type values:** - `behavior` — Judge-scored behavior (name = behavior name, e.g. "Relevance") - `static` — Built-in metrics like "duration" (ms) or "llm_cost" (USD) - `prompt`/`custom` — Prompt or custom scorer by name - `span_attribute` — Arbitrary span attribute key (name = attribute key) - `error` — Span error condition |
| <code style={{whiteSpace:"nowrap"}}>--combine-type</code> | `all`, `any` | yes | — |
| <code style={{whiteSpace:"nowrap"}}>--actions</code> | text | no | JSON object describing what happens when the automation fires. All top-level keys are optional — include only the actions you want configured.  **Shape:**   Slack notifications are configured per-organization in the Judgment UI; pass `"slack"` in `communication_methods` to use them. |
| <code style={{whiteSpace:"nowrap"}}>--cooldown-period</code> | text | no | JSON object describing the minimum wait between triggers. Omit to leave the cooldown unset; if provided, both `value` and `unit` are required.  **Shape:** `{ "value": &lt;number&gt;, "unit": "seconds" \| "minutes" \| "hours" \| "days" }`  Example: `{ "value": 15, "unit": "minutes" }` (at least 15 min between triggers) |
| <code style={{whiteSpace:"nowrap"}}>--trigger-frequency</code> | text | no | JSON object describing the rate-limit window. Omit to leave unset; if provided, all three fields are required.  **Shape:** `{ "count": &lt;number&gt;, "period": &lt;number&gt;, "period_unit": "seconds" \| "minutes" \| "hours" \| "days" }`  Example: `{ "count": 5, "period": 1, "period_unit": "hours" }` (max 5 triggers per 1 hour) |
| <code style={{whiteSpace:"nowrap"}}>-o</code>, <code style={{whiteSpace:"nowrap"}}>--output</code> | `yaml`, `json` | no | Output format. |

**`--conditions` shape**

```json
{
  "metric": {
    "scorer_type": "behavior" | "judge" | "prompt" | "custom" | "static" | "span_attribute" | "error",
    "name": "<scorer or metric name>",
    "threshold": <number | string | null>?
  },
  "comparison": "lt" | "gt" | "eq" | "gte" | "lte" | "fails" | "succeeds" | "chooses" | "detected" | "equals" | "contains" | "exists"
}
```

**`--actions` shape**

```json
{
  "notification": {
    "enabled": <bool>?,
    "communication_methods": ["email" | "slack" | "pagerduty"],
    "email_addresses": ["<addr>", ...]?,
    "pagerduty_config": {"routing_key":"<key>","severity":"critical"|"error"|"warning"|"info"}?
  }?,
  "dataset_addition": {
    "enabled": <bool>?,
    "dataset_name": "<dataset>",
    "metadata_fields": <any>?
  }?,
  "behavior_evaluation": {
    "enabled": <bool>?,
    "behavior_judge_names": ["<judge_name>", ...]
  }?
}
```


### `automations delete`

Delete an automation.

Delete an automation. Requires the admin role.

```bash
judgment automations delete [OPTIONS] [[[ORG_ID] PROJECT_ID] RULE_ID...]
```

**Arguments**

| Name | Required |
| ---- | -------- |
| `[[ORG_ID] PROJECT_ID] RULE_ID` | no |

**Options**

| Flag | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
| <code style={{whiteSpace:"nowrap"}}>--organization-id</code>, <code style={{whiteSpace:"nowrap"}}>--org-id</code> | text | no | Organization ID. Defaults to JUDGMENT_ORG_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--organization</code>, <code style={{whiteSpace:"nowrap"}}>--org</code> | text | no | Organization name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--project-id</code> | text | no | Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--project</code> | text | no | Project name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>-o</code>, <code style={{whiteSpace:"nowrap"}}>--output</code> | `yaml`, `json` | no | Output format. |


### `automations get`

Get an automation by ID.

```bash
judgment automations get [OPTIONS] [[[ORG_ID] PROJECT_ID] RULE_ID...]
```

**Arguments**

| Name | Required |
| ---- | -------- |
| `[[ORG_ID] PROJECT_ID] RULE_ID` | no |

**Options**

| Flag | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
| <code style={{whiteSpace:"nowrap"}}>--organization-id</code>, <code style={{whiteSpace:"nowrap"}}>--org-id</code> | text | no | Organization ID. Defaults to JUDGMENT_ORG_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--organization</code>, <code style={{whiteSpace:"nowrap"}}>--org</code> | text | no | Organization name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--project-id</code> | text | no | Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--project</code> | text | no | Project name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>-o</code>, <code style={{whiteSpace:"nowrap"}}>--output</code> | `yaml`, `json` | no | Output format. |


### `automations list`

List automations.

```bash
judgment automations list [OPTIONS] [[[ORG_ID] PROJECT_ID]...]
```

**Arguments**

| Name | Required |
| ---- | -------- |
| `[[ORG_ID] PROJECT_ID]` | no |

**Options**

| Flag | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
| <code style={{whiteSpace:"nowrap"}}>--organization-id</code>, <code style={{whiteSpace:"nowrap"}}>--org-id</code> | text | no | Organization ID. Defaults to JUDGMENT_ORG_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--organization</code>, <code style={{whiteSpace:"nowrap"}}>--org</code> | text | no | Organization name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--project-id</code> | text | no | Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--project</code> | text | no | Project name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>-o</code>, <code style={{whiteSpace:"nowrap"}}>--output</code> | `table`, `yaml`, `json` | no | Output format. |


### `automations update`

Update an automation.

Update an existing automation. All fields other than the IDs are optional — only supplied fields are applied. Use `active: true/false` to enable or disable without changing other fields. Requires the developer role.

```bash
judgment automations update [OPTIONS] [[[ORG_ID] PROJECT_ID] RULE_ID...]
```

**Arguments**

| Name | Required |
| ---- | -------- |
| `[[ORG_ID] PROJECT_ID] RULE_ID` | no |

**Options**

| Flag | Type | Required | Description |
| ---- | ---- | -------- | ----------- |
| <code style={{whiteSpace:"nowrap"}}>--organization-id</code>, <code style={{whiteSpace:"nowrap"}}>--org-id</code> | text | no | Organization ID. Defaults to JUDGMENT_ORG_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--organization</code>, <code style={{whiteSpace:"nowrap"}}>--org</code> | text | no | Organization name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--project-id</code> | text | no | Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context. |
| <code style={{whiteSpace:"nowrap"}}>--project</code> | text | no | Project name to resolve. |
| <code style={{whiteSpace:"nowrap"}}>--name</code> | text | no | New name for the automation. |
| <code style={{whiteSpace:"nowrap"}}>--description</code> | text | no | New description for the automation. |
| <code style={{whiteSpace:"nowrap"}}>--conditions</code> | text | no | JSON array of rule conditions. Each condition references a named metric/scorer on the project and a comparison. Items are ANDed or ORed together based on `combine_type` (`all` vs `any`).  **Condition shape:**   **Common scorer_type values:** - `behavior` — Judge-scored behavior (name = behavior name, e.g. "Relevance") - `static` — Built-in metrics like "duration" (ms) or "llm_cost" (USD) - `prompt`/`custom` — Prompt or custom scorer by name - `span_attribute` — Arbitrary span attribute key (name = attribute key) - `error` — Span error condition |
| <code style={{whiteSpace:"nowrap"}}>--combine-type</code> | `all`, `any` | no | — |
| <code style={{whiteSpace:"nowrap"}}>--actions</code> | text | no | JSON object describing what happens when the automation fires. All top-level keys are optional — include only the actions you want configured.  **Shape:**   Slack notifications are configured per-organization in the Judgment UI; pass `"slack"` in `communication_methods` to use them. |
| <code style={{whiteSpace:"nowrap"}}>--active</code> | boolean | no | Enable (true) or disable (false) the automation without modifying other fields. |
| <code style={{whiteSpace:"nowrap"}}>--cooldown-period</code> | text | no | JSON 2-tuple `[period, unit]` describing the minimum wait between triggers. Omit to leave unchanged.  **Shape:** `[&lt;period:number&gt;, &lt;unit:"seconds"\|"minutes"\|"hours"\|"days"&gt;]`  Example: `[15, "minutes"]` (at least 15 min between triggers) |
| <code style={{whiteSpace:"nowrap"}}>--trigger-frequency</code> | text | no | JSON 3-tuple `[count, period, unit]` describing the rate-limit window. Omit to leave unchanged.  **Shape:** `[&lt;max_trigger_count:number&gt;, &lt;period:number&gt;, &lt;unit:"seconds"\|"minutes"\|"hours"\|"days"&gt;]`  Example: `[5, 1, "hours"]` (max 5 triggers per 1 hour) |
| <code style={{whiteSpace:"nowrap"}}>-o</code>, <code style={{whiteSpace:"nowrap"}}>--output</code> | `yaml`, `json` | no | Output format. |

**`--conditions` shape**

```json
{
  "metric": {
    "scorer_type": "behavior" | "judge" | "prompt" | "custom" | "static" | "span_attribute" | "error",
    "name": "<scorer or metric name>",
    "threshold": <number | string | null>?
  },
  "comparison": "lt" | "gt" | "eq" | "gte" | "lte" | "fails" | "succeeds" | "chooses" | "detected" | "equals" | "contains" | "exists"
}
```

**`--actions` shape**

```json
{
  "notification": {
    "enabled": <bool>?,
    "communication_methods": ["email" | "slack" | "pagerduty"],
    "email_addresses": ["<addr>", ...]?,
    "pagerduty_config": {"routing_key":"<key>","severity":"critical"|"error"|"warning"|"info"}?
  }?,
  "dataset_addition": {
    "enabled": <bool>?,
    "dataset_name": "<dataset>",
    "metadata_fields": <any>?
  }?,
  "behavior_evaluation": {
    "enabled": <bool>?,
    "behavior_judge_names": ["<judge_name>", ...]
  }?
}
```