Skip to Content
Judgment Labs

Build custom dashboards

Chart your project's traces, judge labels, and spend on a monitoring dashboard, built from SQL by hand or with the Judgment Agent.

A custom dashboard is a board of charts on your project's Monitoring page. Use one to keep the numbers you check every day in one place: trace volume, latency, errors, spend by model, or how often a judge applies a label.

Most charts run one read-only SQL query over your project's data and draw the result. Because a chart is just a query, you can chart anything those tables hold, and you don't have to write the SQL yourself. Two built-in charts, Labels and Label relations, read judge labels directly and have no query to edit. A project can have several dashboards, and each one is a tab.

There are four ways to build one:

PathBest for
TemplatesA working dashboard in one click
The chart editorWriting the SQL and choosing the chart type and columns
The Judgment AgentDescribing the chart you want in plain words
Your coding agentCreating dashboards from Codex, Claude Code, or Cursor

Start a dashboard

A project with no dashboards opens Monitoring on Build your dashboard. Choose a template, or select Create chart to start from scratch.

TemplateCharts
Project overviewTotal traces, p95 latency, error traces, total spend, trace volume, spend by model, and the built-in Labels chart
Cost & usageAverage cost per trace, spend over time, spend by model, and model usage
JudgesThe built-in Labels and Label relations charts, which compare judge labels across traces and sessions

Either way, Judgment creates a dashboard named Overview with those charts on it. Every chart is editable afterward.

To add another dashboard, select New dashboard (the plus on the tab bar). Use the tabs to switch between dashboards.

Set the time range and environments

Two filters at the top of a dashboard apply to every chart on it:

  • Time range: Past 1 Hour, 12 Hours, 24 Hours, 7 Days, 30 Days, or All Time. A chart pinned to its own time range ignores this filter.
  • Environments: limits every chart to the environments you pick.

To change the range a dashboard opens with, choose a range, then select Settings > Use … by default. You can also set it, along with the name and description, under Settings > Dashboard settings.

Add a chart

Select Add chart to open New chart. Choose where to start:

Start fromWhat happens
The agent boxDescribe the chart, for example "Daily errors by tool for the last week". The Judgment Agent builds it beside you.
TemplatesOne click adds a ready-made chart, drawn over your own data, to the dashboard. A toast offers Open in editor.
Built-in chartsOne click adds Labels or Label relations. These have no SQL to edit.
Build it yourselfOpens the chart editor on an empty chart.

Edit a chart

The editor opens on the chart's own page, with the form on the left, a live preview on the right, and the Judgment Agent docked beside them.

Write the query

Pick a Chart type, then write the Query. A Stat chart also takes a Headline query. See Write the SQL.

Map the columns

Set the X axis and Y axis from the columns your query returns. Use Split by on a bar or line chart to draw one series per value. A table takes Columns instead.

Check the preview, then save

Set a time range for the chart if you want one, or leave it on the dashboard's. Select Save, or Add to dashboard for a chart that isn't on one yet. The chart lands in the first free spot on the board.

To start from a template instead, select Templates above the preview. A template replaces the form, and nothing changes until you save.

A chart you haven't saved is a draft. The dashboard shows a drafts button with the count, so a draft you walk away from isn't lost.

Write the SQL

A chart query is a single SELECT over the tables in SQL schema. Name every column you plot with AS. To follow the dashboard's time filter, filter on the {start_time} and {end_time} placeholders:

SELECT toStartOfInterval(started_at, INTERVAL 1 DAY) AS bucket,
       count() AS traces
FROM telemetry.traces
WHERE started_at >= {start_time:DateTime64(6,'UTC')}
  AND ({end_time:Nullable(DateTime64(6,'UTC'))} IS NULL
       OR started_at < {end_time:Nullable(DateTime64(6,'UTC'))})
GROUP BY bucket
ORDER BY bucket

Placeholders only work inside a chart

A query without them ignores the time filter. Outside a chart, such as in the sql_query MCP tool or the SDK, use literal times instead.

Saving runs the query once. It's rejected if it fails or doesn't return a column the chart uses.

Choose how it's drawn

Chart typeWhat it needs
Bar, line, areaAn x-axis column (a category or a time bucket) and a y-axis column. A series column splits it into one series per value.
PieA category column and a value column.
TableThe columns to show, in order.
StatOne headline number over a sparkline. It takes a second query that returns the headline value over the whole time range.

Bar, line, and area charts also have display options. The editor form doesn't show them. Ask the Judgment Agent or use the MCP tools, and the preview and the dashboard draw them:

OptionApplies toEffect
HorizontalBarCategories run down the side. Time-bucketed charts always draw vertical.
StackedBar, lineStacks series into one bar, or draws stacked areas. Use it for series that are parts of a whole.
Value labelsBarPrints each bar's total at its end.
CumulativeLine, areaPlots a running total.
Y-axis titleLine, areaNames the axis. {bucket} reads as the time step, so Latency per {bucket} shows "Latency per hour".
Series colorsAny chart with seriesFixes green or red for up to eight series, such as a success and an error series.

Ask the Judgment Agent about a chart

The Judgment Agent works with charts in three ways:

  • Builds and edits them. Ask for the chart you want in any chat. It saves the chart to a dashboard, or leaves it as a draft for you to review.
  • Cites them. When the agent mentions a chart, it appears as a chip. Hover to preview the chart, or select it to open the chart on its dashboard.
  • Reads them. To ask about a chart you already have, open its tile menu and select Add to agent context. You can also type @ in the composer and pick from Charts.

Manage dashboards and charts

A chart's tile menu:

ActionWhat it does
Open in editorOpens the chart page.
Add to agent contextPuts the chart in the agent's composer.
DuplicateCopies the chart.
RemoveTakes the chart off the dashboard and archives it.

A dashboard's Settings menu:

ActionWhat it does
Dashboard settingsChanges the name, description, and default time range.
DuplicateCopies the dashboard.
Set as default dashboardMakes it the one you open on Monitoring. The default is per person.
ArchiveHides the dashboard.

When a project has archived dashboards, Restore dashboard… brings one back.

Use dashboards from your coding agent

With the Judgment MCP server connected, your agent can list and read dashboards. With Developer access, it can also create and edit dashboards and their charts. The tools are listed under Dashboards.

Use the Judgment MCP tools on the lunch-order-agent project. Call discover_schema,
then create a dashboard named "Reliability" with a line chart of daily error
traces and a horizontal bar chart of failed tool calls by tool name.

Troubleshooting

  • "Couldn't load this chart." The query failed. Open the chart, check the Query, and fix it.
  • "No data yet." The query ran but returned no rows. Widen the time range, or check the environment filter.
  • The chart ignores the time filter. The query is missing the {start_time} and {end_time} placeholders, or the chart is pinned to its own time range.

Next steps

On this page