Query traces with SQL

Run read-only SQL over your project's traces, spans, judge labels and test runs.

Judgment’s virtual SQL (vsql) is a set of read-only tables over one project: traces, spans, judge results, alerts, test config runs, datasets, and judges. You write one SELECT, and the server applies your organization and project. The tables and columns are listed in SQL schema.

There are three ways to run a query:

Ask from your coding agent

With the Judgment MCP server connected, give your agent the question and let it check the schema first:

Use the Judgment MCP tools on the lunch-order-agent project. Call discover_schema,
then use sql_query to find which tools failed most in production this week.
Return the tool name, error count, and one sample error. Don't change anything.

Example queries

These use names from the lunch-order agent. Start with the slowest production requests this week. Each row in telemetry.traces is one request (the root span).

SELECT trace_id, trace_name, started_at, duration_ms
FROM telemetry.traces
WHERE started_at >= now() - INTERVAL 7 DAY
  AND environment_name = 'production' AND duration_ms > 10000
ORDER BY duration_ms DESC, trace_id
LIMIT 20

Next, find which tools fail most, such as a get_menu call that keeps timing out.

SELECT span_name, count() AS errors, any(error_message) AS sample_error
FROM telemetry.spans
WHERE started_at >= now() - INTERVAL 7 DAY AND status = 'error'
GROUP BY span_name ORDER BY errors DESC LIMIT 20

To see cost by model, sum over spans. Cost and token columns live there and are zero when your instrumentation doesn’t report them.

SELECT model, count() AS calls, round(sum(cost_usd), 2) AS total_cost_usd,
       sum(non_cached_input_tokens + cache_read_input_tokens) AS total_input_tokens,
       sum(output_tokens) AS total_output_tokens
FROM telemetry.spans
WHERE started_at >= now() - INTERVAL 7 DAY AND model IS NOT NULL
GROUP BY model ORDER BY total_cost_usd DESC LIMIT 20

To count how users feel by session, read the judge results in telemetry.scores. Here the session judge that asks “How does the user feel about the agent?” is named User sentiment, and its labels are Positive, Negative, and Neutral. A session can be scored more than once, so count distinct sessions.

SELECT value_text AS label, uniqExact(session_id) AS sessions
FROM telemetry.scores
WHERE judge_name_at_evaluation = 'User sentiment' AND evaluation_scope = 'session'
  AND error_message IS NULL AND evaluated_at >= now() - INTERVAL 7 DAY
GROUP BY label ORDER BY sessions DESC

To compare two agent versions, group by agent_version. It comes from the judgment.agent_version attribute you set, for example lunch-agent-v1 and lunch-agent-v2.

SELECT agent_version, count() AS requests, countIf(status = 'error') AS errors,
       quantile(0.5)(duration_ms) AS p50_ms
FROM telemetry.traces
WHERE started_at >= now() - INTERVAL 7 DAY
  AND agent_version IN ('lunch-agent-v1', 'lunch-agent-v2')
GROUP BY agent_version

API and SQL names still say behavior; it means label. To join a result to the label definition, match telemetry.scores.behavior_id_at_evaluation to config.behaviors.behavior_id.

Run SQL from code

Use judgeval 1.3.3+ (Python) or 1.3.4+ (TypeScript), with JUDGMENT_API_KEY and JUDGMENT_ORG_ID set. Your organization needs SDK and HTTP SQL turned on first.

from judgeval import Judgeval

client = Judgeval(project_name="lunch-order-agent")
result = client.sql(
    "SELECT span_name, count() AS errors FROM telemetry.spans "
    "WHERE status = 'error' AND started_at >= now() - INTERVAL 7 DAY "
    "GROUP BY span_name ORDER BY errors DESC LIMIT 20"
)
print(result["rows"])

import { Judgeval } from "judgeval";

const client = await Judgeval.create({ projectName: "lunch-order-agent" });
const result = await client.sql(
  "SELECT span_name, count() AS errors FROM telemetry.spans " +
    "WHERE status = 'error' AND started_at >= now() - INTERVAL 7 DAY " +
    "GROUP BY span_name ORDER BY errors DESC LIMIT 20",
);
console.log(result.rows);

The response has catalog_version, columns (name, type, nullable), rows, row_count, and elapsed_ms. Integers too large for JavaScript arrive as decimal strings. Over HTTP, send {"sql": "..."} to POST https://api.judgmentlabs.ai/v1/projects/{projectId}/sql with the Authorization: Bearer and X-Organization-Id headers from Authentication and projects.

Limits

Each request runs one read-only SELECT of up to 50,000 characters and returns at most 1,000 rows and 5 MiB. Going over either cap returns an error. SDK and HTTP queries share a per-organization rate limit (a burst of 40, refilling 4 per second).

LIMIT caps the rows you get back, not the rows scanned. Narrow by started_at or by trace IDs, and select only the columns you need. If you get QUERY_MEMORY_LIMIT, shorten the time range and try again.