---
title: "Judgment concepts"
description: "Learn how Judgment turns agent traces into production monitoring and offline tests."
sidebar:
  label: "Concepts"
seo:
  title: "Judgment Concepts | Traces, Judges, Behaviors, Datasets, and Tests"
  description: "Learn the core Judgment mental model and the dependency order from agent evidence to production monitoring and offline tests."
---

Judgment turns production evidence into reusable evaluation for live monitoring
and offline tests.

> **Sales assistant example**
>
> Consider a sales assistant that searches past meetings, retrieves a transcript,
> looks up account context, and drafts a follow-up. Judgment records that work,
> evaluates whether it was done correctly, tracks known failure patterns, and
> preserves important cases for testing future changes.

## Shared foundations

### Traces capture what happened

A [trace](/documentation/tracing) records one agent run: model and tool calls,
application logic, inputs, outputs, timing, cost, and errors. Monitoring and
tests use that record as evidence.

> For the sales assistant, one trace can begin with “Summarize my last meeting
> with Acme and draft a follow-up” and end with the completed draft. Meeting
> search, transcript retrieval, account lookup, and generation are child spans.

### Judges produce evaluation results

A [judge](/documentation/judges) defines what to evaluate. Agent Judges use
natural-language rubrics; Code Judges use Python. Each evaluation produces a
[result](/documentation/judges) with a value, reasoning, and evidence.

> The sales assistant might use judges for **Request type**, **Follow-up
> grounding**, and **Meeting search failure**. Together, they distinguish what
> the user wanted, whether the agent succeeded, and whether its environment
> worked.

---

## Online monitoring

```mermaid
flowchart LR
  Trace["Trace"] -->|"evaluated by"| Judge["Judge"]
  Judge -->|"produces"| Result["Result"]
  Result -->|"maps to"| Behavior["Behavior"]
  Behavior -->|"tracked in"| Monitoring["Monitoring"]
```

### Behaviors make judge outputs monitorable

A [behavior](/documentation/judges/behavior-setup) is a binary or categorical
judge output selected for production monitoring. It belongs to its judge;
numeric outputs remain scores.

### Monitoring evaluates production traffic

[Monitoring](/documentation/monitoring) evaluates incoming traces with known
behaviors, shows their rate and volume, and can trigger alerts. Production
Issues surface recurring problems not already encoded by a judge.

> Use a behavior when the criterion is already known, such as an unsupported
> follow-up. Use [Production issues](/documentation/monitoring/production-issues)
> when you want Judgment to discover recurring patterns you have not defined.

---

## Offline testing

```mermaid
flowchart LR
  Trace["Trace"] -->|"saved in"| Dataset["Dataset"]
  Dataset -->|"supplies evidence"| Test["Test run"]
  Judge["Judge"] -->|"evaluates"| Test
  Test -->|"compared in"| Comparison["Comparison"]
```

### Datasets preserve evidence

A [dataset](/documentation/datasets) is a versioned set of examples, traces, or
sessions kept for calibration and testing. Test runs stay pinned to the dataset
version they evaluated.

### Tests evaluate a fixed change

An offline [test](/documentation/tests/offline-tests) applies selected judge
versions to one dataset version, keeping baseline and candidate comparisons
reproducible as those assets change.

> Save traces where the sales assistant used the wrong meeting, then run the
> current and proposed agent against the same cases. The comparison shows
> whether the change fixed the failure without changing the evidence.

## Use the concepts in dependency order

1. Instrument the agent and verify that each trace contains the evidence a
   teammate would need to reconstruct the run.
2. Create judges for the requests, outcomes, or environment conditions you
   need to evaluate.
3. Link known binary or categorical outputs as behaviors for production
   monitoring.
4. Save representative production evidence in datasets and use it in offline
   tests before releasing a change.

Create judges before behaviors. Tests require both judges and a dataset.

---

## Judgment platform

[Judgment Agent](/documentation/judgment-agent) combines the
current platform page with project tools to investigate traces, judges,
behaviors, tests, and production issues. The [Judgment MCP
server](/documentation/mcp-and-agent-tools) provides project access to coding
agents.

---

## Next step

Use [Judgment Agent](/documentation/judgment-agent) to
investigate the trace and judge result from the Quickstart.