---
title: "Cookbook: Data analysis"
description: "Set up judges so the Judgment Agent can answer questions like why users got frustrated last week."
sidebar:
  label: "Cookbook: Data analysis"
seo:
  title: "Cookbook: Data analysis | Judgment Docs"
  description: "Set up judges so the Judgment Agent can answer questions like why users got frustrated last week."
---

Using the Judgment Agent alongside tracing and judges gives you answers to complex questions about your agent, like this one about the [lunch-order agent](/documentation/start/example-agent):

> Identify when users were frustrated in the past week, tracking trends across size of orders and types of restaurants, and let me know what caused their frustration, whether these problems are new or recurring, and if their problem ended up being solved.

This cookbook sets up Judgment to answer that question, and others like it.

If you haven't sent traces yet, start with [Set up tracing](/documentation/connect/set-up-tracing).

## Set up the judges

Each judge below labels every new trace, so the Judgment Agent can look up labels instead of rereading traces. [Labels as indexes](/documentation/agents/labels-as-indexes) explains why. Each judge is a question plus its labels:

| Question the judge answers | Scope | Labels |
|---|---|---|
| How does the user feel about the agent? | Session | Positive, Negative, Neutral |
| How big are orders? | Trace | Small (under $200), Medium ($200 to $500), Large (over $500) |
| Where do users order from? | Trace | One label per cuisine type (Mediterranean, Mexican, Thai, ...) |
| Was the user's original request completed? | Session | Completed, Not completed |

The fastest way to create all four is to ask the Judgment Agent from your project's home page:

```text
Create four judges for this project:
1. "User sentiment": session scoped, with Positive, Negative, Neutral.
   Judge how the user feels about the agent across the whole conversation.
2. "Order size": trace scoped, with Small (under $200), Medium ($200 to $500),
   Large (over $500), based on the estimate_order total.
3. "Cuisine": trace scoped, with one choice per cuisine our users order.
4. "Request completed": session scoped, with Completed and Not completed. Not completed
   when the user's original request was never fulfilled, including when the agent
   dropped an option without saying so.
Turn on online monitoring for production traffic.
```

The agent asks before it creates anything. You can also create each judge yourself with **New Judge** in **Judges**; see [Create a judge](/documentation/judges/create-a-judge).

Then check **Run settings** on each judge's **Details** tab:

- **Online monitoring** is on and the version is published to production.
- **Session scoring** is on for the two session judges.
- **Environments** includes the environment your production traffic uses.

[Run a judge on live traffic](/documentation/judges/run-on-live-traffic) covers every run setting.

## Let traffic come in

Judges label traces as they arrive. They don't go back over traces you already have, so give it a day or a week of real traffic before you ask a question that spans a week.

While you wait, open a few labeled traces and check the calls. If **User sentiment** marks an obviously annoyed user as **Neutral**, fix the rubric now, before you build analysis on top of it. [Align a judge](/documentation/judges/align-a-judge) is the structured way to do this.

## Ask the question

Open the Judgment Agent and paste the question from the top of this page. **Deep Research** mode is a good fit for questions like this one, which take several queries to answer.

{/* SCREENSHOT: Judgment Agent answering the frustration question with a chart and cited traces, dark mode */}

Here's roughly what the agent does:

1. Finds sessions labeled **Negative** by **User sentiment** in the past week.
2. Breaks them down by the **Order size** and **Cuisine** labels on their traces, so you can see whether frustration clusters on Large orders or on one cuisine.
3. Checks **Request completed** on the same sessions to see whether the problem ended up being solved.
4. Opens a few of those traces to find the cause, like Cedar Grill's `get_menu` timing out and the agent dropping it without telling the user.
5. Compares with earlier weeks to tell you whether it's new or recurring.

Under the hood, the agent reads labels with SQL. Judgment's virtual SQL (vsql) keeps every judge result in `telemetry.scores`. You can run the same queries from Codex, Claude Code or Cursor with the MCP `sql_query` tool. See [Query with SQL](/documentation/agents/query-with-sql).

## Track trends over time

A question like this is usually worth asking every week. There are two ways to keep it running:

- Save the question as a Scheduled Task's instructions, set it to repeat weekly, and turn on **Post to Slack**. See [Scheduled Tasks](/documentation/agents/scheduled-tasks).
- If you want to hear about every frustrated session as it happens, add an action on **User sentiment** that alerts when it applies **Negative**. See [Judge actions](/documentation/judges/judge-actions).

If a cause shows up week after week, check [Issues](/documentation/issues/overview). Detection looks for recurring problems on its own, without any judges, and may already be tracking it.

## What's manual

- You decide which questions deserve a judge. Judgment can suggest judges (**Suggest judges to create** in **New Judge**), but you pick.
- Judges only label new traffic, so a new question needs time to collect data.
- Fixing the cause is up to you and your coding agent. Judgment shows you where the problem is and which traces prove it.