What's new

Changes to Judgment, by month.

An agent reads your recent traces and turns recurring failures into Issues, each with a severity, a diagnosis and the traces behind it. It needs traces only, no judges. You review each Issue, hand it to Codex, Claude Code, Cursor or Devin with Fix with, and choose Monitor with Judge or Dismiss issue. Detection can run on a schedule and post a report to Slack after each run. See Find issues.

Project setup

New projects open a five-step setup on the project home: Tracing, Issues, Judges, Slack and Judgment Agent. As your first traces arrive, Judgment starts an Issues run and creates up to five judges from them. See the Quickstart.

Tests

Tests run your agent over a set of test inputs, Judgment’s judges grade each trace, and you compare runs. See Test changes.

Query with SQL

Query traces with Judgment’s virtual SQL (vsql) through the Judgment Agent and the MCP tools sql_query and discover_schema. The SDK adds client.sql() in judgeval 1.3.3 (Python) and 1.3.4 (TypeScript). See Query with SQL.

Scheduled Tasks

Background agents are now Scheduled Tasks, under Scheduled in the sidebar. A task is a Judgment Agent chat that runs on a schedule and can post to Slack. Start from scratch, from a template, or let Judgment suggest one. See Scheduled Tasks.

Agent Knowledge

Project memory moved to Agent Knowledge in project Settings. Files there are shared by the Judgment Agent in chats and Scheduled Tasks, and files under skills/ become slash commands. See Agent Knowledge.

Labels

The values a judge applies are now called labels. API and SQL names still say behavior (for example behavior_id). See Labels as indexes.

Deprecated

The CLI and prompt management are deprecated. Old CLI documentation links redirect to the pages that replace them.