SQL

Query Judgment project data through MCP for agents or the Judgeval SQL API for scripts and applications.

Use SQL to query traces, spans, scores, datasets, and test results. Judgment exposes a virtual schema: documented tables and columns that abstract the underlying storage. Agents can discover this schema and write SQL without knowing physical table names or storage details.

Judgment parses and validates incoming queries, rejects writes and unsupported operations, and translates allowed queries through the virtual schema. Only a single read-only SELECT is accepted. The server enforces organization and project scope, so agents can focus on discovering the schema and querying data.

JQL is deprecated for new integrations. Choose the access path for your workflow:

Agent access through MCP

Connect the MCP server, then ask your agent:

Use Judgment MCP to query my project. Call discover_schema first, then use
sql_query to find the 20 most recent traces that took more than 10 seconds
in the last seven days. Return trace IDs, names, start times, and duration_ms.
Do not make any changes.

The agent should use this SQL after checking the current catalog:

SELECT trace_id, trace_name, started_at, duration_ms
FROM telemetry.traces
WHERE started_at >= now() - INTERVAL 7 DAY
  AND duration_ms > 10000
ORDER BY started_at DESC, trace_id
LIMIT 20

Success returns up to 20 matching traces; an empty rows array means no traces matched. Follow the connection’s advertised tool schema for organization and project arguments. See the MCP SQL reference for scope, schema discovery, and query limits.

Programmatic access

Use client.sql(...) to run queries from Python or TypeScript. sql_query is the MCP tool name; both SDKs use sql.

Configure JUDGMENT_API_KEY and JUDGMENT_ORG_ID as described in Authentication, and select an existing project. SQL reads require at least the viewer role.

Public SDK/API queries must also be enabled for your organization. If the endpoint returns 404, verify the project ID and organization, then contact Judgment to confirm access. MCP SQL does not require this separate opt-in.

Python

from judgeval import Judgeval

client = Judgeval(project_name="my-project")
print(client.discover_schema())
result = client.sql("""
    SELECT trace_id, trace_name, started_at, duration_ms
    FROM telemetry.traces
    WHERE started_at >= now() - INTERVAL 7 DAY
      AND duration_ms > 10000
    ORDER BY started_at DESC, trace_id
    LIMIT 20
""")
print(result["rows"])

TypeScript

import { Judgeval } from "judgeval";

const client = await Judgeval.create({ projectName: "my-project" });
console.log(await client.discoverSchema());
const result = await client.sql(`
    SELECT trace_id, trace_name, started_at, duration_ms
    FROM telemetry.traces
    WHERE started_at >= now() - INTERVAL 7 DAY
      AND duration_ms > 10000
    ORDER BY started_at DESC, trace_id
    LIMIT 20
`);
console.log(result.rows);

Both TypeScript methods accept { signal: abortController.signal } as their options argument for cancellation.

See the Python client reference and TypeScript client reference for method signatures and return values.

Schema and results

Python’s client.discover_schema() and TypeScript’s client.discoverSchema() return a Markdown string containing the same generated reference as MCP discover_schema: published tables, column types and descriptions, row semantics, SQL examples, and query limits. It fetches the server’s catalog using your credentials and contains no project data. Schema discovery requires organization viewer access, but no resolved project or public query opt-in, and does not consume the public query quota.

client.sql(...) sends the SQL text to the public endpoint with the client’s credentials and resolved project. The response contains catalog_version, columns (each with name, type, and nullable), rows, row_count, and elapsed_ms. Integers outside JavaScript’s safe range arrive as exact decimal strings, including nested values; use int(value) in Python or BigInt(value) in TypeScript when needed.

HTTP API

Fetch the same schema reference with GET /v1/sql/schema, using the Authorization: Bearer ... and X-Organization-Id headers. The JSON response is {"schema": "...Markdown reference..."}; no project ID or request body is needed.

Use POST /v1/projects/{projectId}/sql for direct HTTP access from any language. The request body has one field, sql:

curl --fail-with-body \
  "https://api.judgmentlabs.ai/v1/projects/${JUDGMENT_PROJECT_ID}/sql" \
  -H "Authorization: Bearer ${JUDGMENT_API_KEY}" \
  -H "X-Organization-Id: ${JUDGMENT_ORG_ID}" \
  -H "Content-Type: application/json" \
  --data '{"sql":"SELECT trace_id, trace_name FROM telemetry.traces WHERE started_at >= now() - INTERVAL 7 DAY ORDER BY started_at DESC, trace_id LIMIT 20"}'

Set JUDGMENT_PROJECT_ID to the existing project’s ID. The API checks that the project belongs to the authenticated organization. Keep credentials in your environment or secret manager, outside SQL text and source control.

Migrate from JQL

Existing Python and TypeScript JQL calls remain available. Migrate each query and its result consumer together:

To preserve JQL’s session scope for span reads, resolve the session’s trace IDs from telemetry.traces, then filter telemetry.spans by those trace IDs.

SQL text must contain a non-whitespace character, be at most 50,000 characters, and contain one read-only SELECT. Results are capped at 1,000 rows and 5 MiB; exceeding a cap returns an error. The SDK/API uses the public query rate limit.

Select only needed fields and narrow by time or trace/span IDs. LIMIT does not bound scan, grouping, sort, or join memory. On QUERY_MEMORY_LIMIT, narrow the scope or simplify the query before retrying. Use SDK schema discovery or MCP schema discovery for the current tables, columns, and query guidance.

For integrations still using builders, the legacy JQL reference remains available.

Schema reference

This reference is generated from the same catalog returned by MCP discover_schema, Python client.discover_schema(), and TypeScript client.discoverSchema().

Judgment SQL (telemetry.v1)

One read-only SELECT against these virtual tables; host supplies org/project scope; SQL cannot override it.

Memory: prefer narrow time ranges or trace/span IDs and only needed fields. Small LIMITs help inspection but do not bound scan, grouping, sort or join memory. Run broad scans sequentially. On QUERY_MEMORY_LIMIT, narrow scope or simplify before retrying.

telemetry.traces

One latest root record per trace_id

telemetry.spans

One latest record per (trace_id, span_id)

telemetry.scores

One retained evaluation result

telemetry.trace_tags

One tag assignment per (trace_id, tag)

telemetry.alerts

One alert generated by an automation rule

evaluations.runs

One current evaluation test run

evaluations.items

One latest scorer result per run, example, judge, and judge version

datasets.datasets

One current dataset

datasets.examples

One dataset membership per example, including its version interval

config.judges

One judge with a production version per judge_id

config.behaviors

One current behavior value per behavior_id

telemetry.offline_traces

Same columns and payload limits as telemetry.traces; offline records only.

telemetry.offline_spans

Same columns and payload limits as telemetry.spans; offline records only.

Query guidance

• Offline evidence: use telemetry.offline_traces and telemetry.offline_spans for offline trace IDs, including evaluations.items.offline_trace_id. These have the live tables’ column shapes and payload limits, but never include live rows. Offline telemetry supports trace scope, not live session scope.
• Population: traces = root executions; spans = operations; sessions group traces. Count traces from telemetry.traces. State the time range; use all history only when requested. Run broad scans sequentially. Explore cardinality with uniqCombined64; use uniqExact for required exact totals.
• Session elapsed_ms = (latest root end - earliest root start), including gaps/overlaps, not summed durations. Aggregate ALL roots before filtering earliest start with inclusive bounds. Usage includes ALL spans with that session ID, even outside the window. Filter IDs in WHERE, grouped metrics in HAVING/outer SELECT. Convert legacy latency_ns thresholds to ms by /1000000. Page by metric plus session_id.
• Session search: equality/IN for IDs; ILIKE for substring/prefix/suffix. Behavior filters: rank session scores per (session_id, judge_id) by evaluated_at DESC, score_id DESC; keep latest binary/categorical rows with no error and non-null behavior_id_at_evaluation. Filter sessions via IN, avoiding multiplied usage. Legacy behaviors:any requires ALL requested IDs: group matches by session_id, require count(DISTINCT behavior_id_at_evaluation) = distinct requested ID count.
• Sum span cost/tokens once BEFORE tag/score joins. Recorded totals may include duplicate wrapper usage; missing usage = zero, not evidence of free calls.
• Scores are results, not unique entities; filter evaluation_scope. Latest behavior inspection ranks by evaluated_at DESC, score_id DESC per entity/judge BEFORE filtering deleted behaviors. Retain errors despite missing metadata; never fall back to older results after deletion/error. History: fetch separately for current-result judge IDs, group by judge_id, paginate by evaluated_at/score_id. Grouped session trace IDs retain duplicates, sorted. value_text is normalized SQL text.
• Historical behavior_id_at_evaluation/judge names differ from current config metadata; config.judges contains production versions only. Binary false need not mean failure: interpret rubric/behavior meaning. Keep judge versions separate in offline comparisons. Use dedicated tools for full judge definitions, eligible behavior denominators, health/issues and other absent data; sampled score fractions are not production behavior rates.
• Dataset version N: version_added <= N AND (version_removed IS NULL OR version_removed > N). Current = no removal version. LEFT JOIN counts must exclude null example IDs to preserve empty datasets. Evaluation items count scorer results; distinct example IDs count examples.
• Separate aggregates from limited detail pages. Order by a unique tie-breaker; continue ORDER BY started_at DESC, trace_id DESC with (started_at < last_time OR (started_at = last_time AND trace_id < last_id)), alongside existing filters.
• Attributes: use bounded substringUTF8 windows and lengthUTF8 for total characters. SQL mapContains distinguishes missing keys from present empty strings; direct lookup returns empty for both. No row = missing span; mapKeys lists keys. substringUTF8 uses 1-based Unicode code points: after 1–2000, start at 2001; never reuse byte, JS UTF-16 or tool offsets. Use small literal LIMITs; narrow/page capped results. Replace example IDs with real IDs and include citation IDs.
• Use sql_query for trace searches. Calibration review state is not exposed; unreviewed calibration filtering is unavailable.

Examples

Session timing and usage

WITH session_roots AS (
  SELECT session_id, min(started_at) AS started_at, max(ended_at) AS ended_at,
         count() AS trace_count
  FROM telemetry.traces
  WHERE session_id IS NOT NULL
  GROUP BY session_id
), session_usage AS (
  SELECT session_id, sum(cost_usd) AS cost_usd,
         sum(non_cached_input_tokens + cache_read_input_tokens + cache_creation_input_tokens) AS input_tokens,
         sum(output_tokens) AS output_tokens
  FROM telemetry.spans
  WHERE session_id IS NOT NULL
  GROUP BY session_id
)
SELECT r.session_id, r.trace_count, r.started_at, r.ended_at,
       dateDiff('microsecond', r.started_at, r.ended_at) / 1000.0 AS elapsed_ms,
       coalesce(u.cost_usd, 0) AS cost_usd,
       coalesce(u.input_tokens, 0) AS input_tokens,
       coalesce(u.output_tokens, 0) AS output_tokens
FROM session_roots AS r
LEFT JOIN session_usage AS u USING (session_id)
ORDER BY r.started_at DESC, r.session_id DESC
LIMIT 50

Current example counts by dataset

SELECT d.dataset_id, d.name, countIf(e.example_id IS NOT NULL AND e.version_removed IS NULL) AS current_examples
FROM datasets.datasets AS d
LEFT JOIN datasets.examples AS e USING (dataset_id)
GROUP BY d.dataset_id, d.name
ORDER BY current_examples DESC
LIMIT 50