SQL

SQL methods and schema reference for the Judgeval Python SDK.

Query project data with SQL through the Judgeval Python SDK. The virtual schema below abstracts the underlying storage. Judgment validates incoming queries, permits one read-only SELECT, and enforces organization and project scope.

Use sql() to run a query and discover_schema() to fetch the schema. See the SQL guide for setup and authentication.

Schema reference

This reference is generated from the catalog used by SDK schema discovery and MCP discover_schema. It contains no project data.

Judgment SQL (telemetry.v1)

One read-only SELECT against these virtual tables; host supplies org/project scope; SQL cannot override it.

Memory: prefer narrow time ranges or trace/span IDs and only needed fields. Small LIMITs help inspection but do not bound scan, grouping, sort or join memory. Run broad scans sequentially. On QUERY_MEMORY_LIMIT, narrow scope or simplify before retrying. ClickHouse types: wrap nullable columns before JSON/array functions, e.g. JSONExtractArrayRaw(ifNull(output, ‘’), ‘key’); write timestamps in full, e.g. started_at >= ‘2026-09-16 06:00:00’, or use parseDateTime64BestEffort(‘2026-09-16T06:00:00Z’, 6, ‘UTC’) for ISO. Correlated subqueries are unsupported; JOIN a grouped subquery instead.

telemetry.traces

One latest root record per trace_id

telemetry.spans

One latest record per (trace_id, span_id)

telemetry.scores

One retained evaluation result

telemetry.trace_tags

One tag assignment per (trace_id, tag)

telemetry.judge_reviews

Latest human review per judge and trace

telemetry.alerts

One alert generated by an automation rule

telemetry.model_usage

LLM usage per bucket, environment, and model; bucket width follows the request window: minute ≤150h, hour ≤3600h, else day (hour with no window)

telemetry.latency

Average duration per bucket, kind, and name across environments; buckets as in telemetry.model_usage. Per environment: use telemetry.spans

evaluations.runs

One legacy test-config run; test-suite results aren’t in SQL

evaluations.items

Legacy test-config result per run, example, judge, judge version; not test suites

datasets.datasets

One current dataset

datasets.examples

One dataset membership per example, including its version interval

config.judges

One judge with a production version per judge_id

config.behaviors

One current label definition per behavior_id

Examples

Session timing and usage

WITH session_roots AS (
  SELECT session_id, min(started_at) AS started_at, max(ended_at) AS ended_at,
         count() AS trace_count
  FROM telemetry.traces
  WHERE session_id IS NOT NULL
  GROUP BY session_id
), session_usage AS (
  SELECT session_id, sum(cost_usd) AS cost_usd,
         sum(non_cached_input_tokens + cache_read_input_tokens + cache_creation_input_tokens) AS input_tokens,
         sum(output_tokens) AS output_tokens
  FROM telemetry.spans
  WHERE session_id IS NOT NULL
  GROUP BY session_id
)
SELECT r.session_id, r.trace_count, r.started_at, r.ended_at,
       dateDiff('microsecond', r.started_at, r.ended_at) / 1000.0 AS elapsed_ms,
       coalesce(u.cost_usd, 0) AS cost_usd,
       coalesce(u.input_tokens, 0) AS input_tokens,
       coalesce(u.output_tokens, 0) AS output_tokens
FROM session_roots AS r
LEFT JOIN session_usage AS u USING (session_id)
ORDER BY r.started_at DESC, r.session_id DESC
LIMIT 50

Current example counts by dataset

SELECT d.dataset_id, d.name, countIf(e.example_id IS NOT NULL AND e.version_removed IS NULL) AS current_examples
FROM datasets.datasets AS d
LEFT JOIN datasets.examples AS e USING (dataset_id)
GROUP BY d.dataset_id, d.name
ORDER BY current_examples DESC
LIMIT 50