---
title: "Run a judge on live traffic"
description: "Turn on online monitoring under Run settings, choose which traffic the judge evaluates, and run a judge on traces you already have."
sidebar:
  label: "Run on live traffic"
seo:
  title: "Run a judge on live traffic | Judgment Docs"
  description: "Turn on online monitoring under Run settings, choose which traffic the judge evaluates, and run a judge on traces you already have."
---

A judge only labels live traffic after you turn on online monitoring. Everything on this page is under **Run settings** on the judge's **Details** tab, except [running a judge on traces you already have](#run-a-judge-on-traces-you-already-have).

## Turn on online monitoring

1. Open **Judges**, pick the judge, and open **Run settings** on the **Details** tab.
2. Turn on **Online monitoring** ("Automatically evaluate incoming traces").
3. Set the options below, then select **Save** and **Publish to production**.

Online monitoring runs the judge's production version. A draft doesn't label anything until you publish or promote it.

**Run settings** appears once the judge exists. For a Code judge you can also set these options through MCP; see [Code judge bundle](/documentation/reference/code-judge-bundle).

If the judge isn't labeling, **Run settings** says so: "This judge is not tagging incoming traces." That appears when monitoring is off or the sampling rate is 0%.

{/* SCREENSHOT: judge Details tab, Run settings with Online monitoring on and Environments, Run conditions, Sampling rate, Session scoring visible, dark mode */}

## Choose which traffic gets judged

### Environments

Select the environments to evaluate, or all of them. You can't save with online monitoring on until you select them. A judge scoped to `production` ignores your local runs.

Traces without an environment show as **No environment**. To set environments in your code, see [Add context](/documentation/connect/add-context).

### Run conditions

Leave them empty to evaluate every trace. Add conditions to evaluate only the traces that match:

- **Duration (seconds)** and **LLM cost (USD)** compare the trace's total with a number (`>`, `≥`, `<`, `≤`, `=`).
- **Error** checks the trace's error: **exists**, **equals**, **does not equal**, or **contains** an error message.
- A **Span group** checks one span. Under **Span scope**, select **Root span** or **Any single span**, then add **Span Name** or **Span Attribute** conditions (with an **Attribute Key**).

Every condition in a span group must match the same span. Combine conditions and span groups with AND, OR, and parentheses.

For the [lunch-order agent](/documentation/start/example-agent), a judge about menu failures could use a span group with **Any single span** and **Span Name** equals `get_menu`, so it only runs on traces that looked up a menu.

> **Warning**
>
> A span group set to **Root span** only checks the trace's root span. A nested span name, such as the `get_menu` tool call, never matches it, and the judge never runs. Use **Any single span** for spans nested anywhere in the trace.

### Sampling rate

The sampling rate is the percentage of incoming traces (or sessions, with session scoring on) to evaluate, from 0 to 100%. Lower it to control cost on high-volume projects. Judge runs use Judgment Units. See [Billing and plans](/documentation/administration/billing-and-plans).

### Session scoring

Turn on **Session scoring** to evaluate the whole session instead of each trace. Then select when a session gets evaluated:

| Option | The session is evaluated |
| --- | --- |
| **After inactivity** | After a number of minutes with no new traces |
| **After new traces** | After a number of new traces that match the run conditions and sampling rate |
| **Either** | Whichever happens first |

Use session scoring for questions that play out over a conversation, like "How does the user feel about the agent?" Condition judges can't use session scoring.

## Run a judge on traces you already have

Online monitoring only labels traces that arrive after you turn it on. To label traces or sessions you already have:

1. Open the traces (or sessions) table and select the ones to evaluate.
2. Select **Run Judge**.
3. Pick the judge and select **Run Judge**.

The list only shows judges that match what you selected: trace judges for traces, and judges with session scoring on for sessions. The judge's production version runs on each item. Its environments, run conditions and sampling rate still apply, so a trace those settings exclude isn't evaluated. These runs don't send the judge's [actions](/documentation/judges/judge-actions).

To evaluate a specific span from your code instead, call `Tracer.async_evaluate` with the judge's name inside an observed function. The evaluation is queued when the span ends and runs in Judgment:

```python
from judgeval import Tracer

Tracer.init(project_name="lunch-order-agent")


@Tracer.observe(span_type="agent", span_name="recommend_order")
def recommend_order(request: str) -> str:
    recommendation = "Cedar Grill for 30 people, total $480."
    Tracer.async_evaluate(
        "Order size",
        {"input": request, "actual_output": recommendation},
    )
    return recommendation


print(recommend_order("Lunch for 30 people on Friday"))
Tracer.shutdown()
```

## If no labels appear

- Check that **Online monitoring** is on and the version you edited is the production version.
- Check the sampling rate is above 0%.
- Check the environment scope matches the environment your traces are sent with.
- Check the run conditions. A span group set to **Root span** never matches a nested span; switch it to **Any single span**.
- Labels apply to traces that arrive after monitoring is on. For older traces, [run the judge on them](#run-a-judge-on-traces-you-already-have).
- With session scoring, a session isn't evaluated until the inactivity window or trace count is reached.