Skip to Content
Judgment Labs

Changelog

New features and changes in Judgment.

October 2026

  1. Tests

    Tests run your agent over a set of test inputs, Judgment's judges grade each trace, and you compare runs.

    Set up tests

  2. Judge actions

    Every judge can now send an email, post to Slack, add the trace to a dataset or run more judges when it returns the labels you pick, with a cooldown. Automations are now judge actions.

    Set up judge actions

  3. Scheduled Tasks

    Background agents are now Scheduled Tasks, under Scheduled in the sidebar. A task is a Judgment Agent chat that runs on a schedule and can post to Slack. Start from scratch, from a template, or let Judgment suggest one.

    Schedule a task

  4. Issues

    An agent reads your recent traces and turns recurring failures into Issues, each with a severity, a diagnosis and the traces behind it. It needs traces only, no judges. You review each Issue, hand it to Codex, Claude Code, Cursor or Devin with Fix with, and choose Monitor with Judge or Dismiss issue. Detection can run on a schedule and post a report to Slack after each run.

    Find issues in your traces

  5. Query with SQL

    Query traces with Judgment's virtual SQL (vsql) through the Judgment Agent and the MCP tools sql_query and discover_schema. The SDK adds client.sql() in judgeval 1.3.3 (Python) and 1.3.4 (TypeScript).

    Query your traces with SQL

  6. Project setup

    New projects open a five-step setup on the project home: Tracing, Issues, Judges, Slack and Judgment Agent. As your first traces arrive, Judgment starts an Issues run and creates up to five judges from them.

    Follow the quickstart

  7. Agent Knowledge

    Project memory moved to Agent Knowledge in project Settings. Files there are shared by the Judgment Agent in chats and Scheduled Tasks, and files under skills/ become slash commands.

    Add agent knowledge

  8. Labels

    The values a judge applies are now called labels. API and SQL names still say behavior (for example behavior_id).

    Use labels as indexes

  9. Deprecated

    The CLI and prompt management are deprecated. Old CLI documentation links redirect to the pages that replace them.