judgment automations CLI Reference

The Judgment CLI is deprecated and no longer under active development. This reference is retained for existing users. See the CLI deprecation notice for supported alternatives.

Manage automations (rules) that fire actions when metrics match conditions.

Command	Description
`automations create`	Create an automation.
`automations delete`	Delete an automation.
`automations get`	Get an automation by ID.
`automations list`	List automations.
`automations update`	Update an automation.

Name	Required
`[[ORG_ID] PROJECT_ID] NAME`	no

Name	Required
`[[ORG_ID] PROJECT_ID]`	no

Update an existing automation. All fields other than the IDs are optional — only supplied fields are applied. Use active: true/false to enable or disable without changing other fields. Requires the developer role.

judgment automations update [OPTIONS] [[[ORG_ID] PROJECT_ID] RULE_ID...]

Arguments

Name	Required
`[[ORG_ID] PROJECT_ID] RULE_ID`	no

Options

Flag	Type	Required	Description
`--organization-id`, `--org-id`	text	no	Organization ID. Defaults to JUDGMENT_ORG_ID or saved context.
`--organization`, `--org`	text	no	Organization name to resolve.
`--project-id`	text	no	Project ID. Defaults to JUDGMENT_PROJECT_ID or saved context.
`--project`	text	no	Project name to resolve.
`--name`	text	no	New name for the automation.
`--description`	text	no	New description for the automation.
`--conditions`	text	no	JSON array of rule conditions. Each condition references a named metric/scorer on the project and a comparison. Items are ANDed or ORed together based on `combine_type` (`all` vs `any`). Condition shape: Common scorer_type values: - `behavior` — Judge-scored behavior (name = behavior name, e.g. "Relevance") - `static` — Built-in metrics like "duration" (ms) or "llm_cost" (USD) - `prompt`/`custom` — Prompt or custom scorer by name - `span_attribute` — Arbitrary span attribute key (name = attribute key) - `error` — Span error condition
`--combine-type`	`all`, `any`	no	—
`--actions`	text	no	JSON object describing what happens when the automation fires. All top-level keys are optional — include only the actions you want configured. Shape: Slack notifications are configured per-organization in the Judgment UI; pass `"slack"` in `communication_methods` to use them.
`--active`	boolean	no	Enable (true) or disable (false) the automation without modifying other fields.
`--cooldown-period`	text	no	JSON 2-tuple `[period, unit]` describing the minimum wait between triggers. Omit to leave unchanged. Shape: `[<period:number>, <unit:"seconds"\|"minutes"\|"hours"\|"days">]` Example: `[15, "minutes"]` (at least 15 min between triggers)
`--trigger-frequency`	text	no	JSON 3-tuple `[count, period, unit]` describing the rate-limit window. Omit to leave unchanged. Shape: `[<max_trigger_count:number>, <period:number>, <unit:"seconds"\|"minutes"\|"hours"\|"days">]` Example: `[5, 1, "hours"]` (max 5 triggers per 1 hour)
`-o`, `--output`	`yaml`, `json`	no	Output format.

--conditions shape

{
  "metric": {
    "scorer_type": "behavior" | "judge" | "prompt" | "custom" | "static" | "span_attribute" | "error",
    "name": "<scorer or metric name>",
    "threshold": <number | string | null>?
  },
  "comparison": "lt" | "gt" | "eq" | "gte" | "lte" | "fails" | "succeeds" | "chooses" | "detected" | "equals" | "contains" | "exists"
}

--actions shape

{
  "notification": {
    "enabled": <bool>?,
    "communication_methods": ["email" | "slack" | "pagerduty"],
    "email_addresses": ["<addr>", ...]?,
    "pagerduty_config": {"routing_key":"<key>","severity":"critical"|"error"|"warning"|"info"}?
  }?,
  "dataset_addition": {
    "enabled": <bool>?,
    "dataset_name": "<dataset>",
    "metadata_fields": <any>?
  }?,
  "behavior_evaluation": {
    "enabled": <bool>?,
    "behavior_judge_names": ["<judge_name>", ...]
  }?
}

judgment automations

Commands

`automations create`

`automations delete`

`automations get`

`automations list`

`automations update`

On this page