MCP tools

Reference for every currently registered Judgment data MCP tool, category, access level, and input field.

The Judgment data MCP groups its tools by product area. The schema returned by your client’s tools/list call is authoritative for the tools and fields available to that connection.

Scope and access

SQL and schema discovery are available without a separate rollout opt-in. Tool availability still depends on the connection’s organization, project, role, and feature permissions.

• An unbound API-key connection supplies organization_id to every organization- or project-scoped tool and project_id to every project-scoped tool. API-key connections with an X-Organization-Id header can omit organization_id.
• OAuth authorization binds the connection to an organization and optionally a project. The MCP server removes bound scope fields from the model-facing schema and injects them server-side. Project-bound connections also hide account- or organization-level tools that fall outside the selected project.
• discover_schema requires organization scope and at least the viewer role, but returns no project data. Call it with {} on an organization-bound connection or with organization_id on an unbound connection.
• Read tools inspect state. Write · developer and Write · admin require at least that organization role. A plain Write tool has more specific authorization in its description or the underlying resource.
• The Inputs column lists the registered field names beyond the common scope fields. Use tools/list for which fields are required and their exact nested schemas.

SQL and schema discovery

Use SQL to query your project’s traces, spans, scores, datasets, and test results through a virtual schema that abstracts the underlying storage. The server validates incoming queries, permits only read-only access, and enforces organization and project scope. MCP is the recommended access path for agents. For scripts and applications, see programmatic SQL access.

Query workflow

1. Call discover_schema to get the current schema. Use {} on an organization-bound connection or { "organization_id": "YOUR_ORGANIZATION_ID" } on an unbound connection. Use the table and column names returned by this tool.
2. Call sql_query with a sql string and any scope arguments your connection requests. Use list_organizations and list_projects to find IDs when those tools are available on your connection.
3. Read the returned rows. The response also contains catalog_version, columns (each with name, type, and nullable), row_count, and elapsed_ms.

For example, these sql_query arguments find the 20 slowest traces from the last day on an API-key connection without bound organization or project scope:

{
  "organization_id": "YOUR_ORGANIZATION_ID",
  "project_id": "YOUR_PROJECT_ID",
  "sql": "SELECT trace_id, trace_name, started_at, duration_ms FROM telemetry.traces WHERE started_at >= now() - INTERVAL 1 DAY ORDER BY duration_ms DESC, trace_id LIMIT 20"
}

For organization-scoped OAuth, omit organization_id; for project-scoped OAuth, omit both scope arguments. Follow the input schema advertised by your connection.

To inspect a trace’s span payloads, use a focused query such as:

SELECT span_id, parent_span_id, span_name, input, output, attributes
FROM telemetry.spans
WHERE trace_id = 'YOUR_TRACE_ID'
ORDER BY started_at, span_id
LIMIT 20

Query limits and data semantics

• Submit one read-only SELECT, at most 50,000 characters. Queries are scoped to your authorized project.
• Results are limited to 1,000 rows and 5 MiB. Select the columns you need, filter by time or IDs, and use a small LIMIT when exploring.
• Large fields such as input, output, attributes, and JSON payloads have no additional SQL restrictions. Prefer narrow scopes and small payload reads. The default query memory limit is 750 MiB, configurable per deployment. A small LIMIT does not bound scan, grouping, sort, or join memory. On QUERY_MEMORY_LIMIT, narrow the scope or simplify the query before retrying.
• Query telemetry.traces for root executions and telemetry.spans for operations, cost, and token usage. Sum span usage before joining tags or scores to avoid counting it multiple times.
• Sessions are derived by grouping traces by session_id; there is no telemetry.sessions table. Session elapsed time runs from the earliest root start to the latest root end, including gaps and overlaps. Follow the schema’s session guidance for time filters, usage, and behavior results.
• Query telemetry.scores for evaluation results and filter evaluation_scope when selecting span, trace, or session scores. Use get_behavior_detail for eligible-population behavior rates; sampled score fractions are not equivalent.
• Test execution traces use telemetry.traces and telemetry.spans. The legacy evaluations.items.offline_trace_id column identifies an execution trace in these shared tables.

Migrating from retired read tools

The following tool names are no longer callable. Replace them with discover_schema followed by sql_query:

If your client still advertises retired names, refresh its tool list or reconnect. The available tools below include the retained behavior-detail, Trace Health, automation, mapper, dataset, and memory tools.

Organizations

Projects

These tools are organization-scoped.

Traces

Behaviors

Judges

Datasets

Tests

The offline-test interface has nine tools. Use list_test_runs to find runs, get_test_run for one run’s metadata and status, and get_test_run_items for paginated per-example results. Aggregate score charts remain available on the test run page; use sql_query against evaluations.items for custom score distributions and aggregates. Keep judge versions separate when comparing results.

Automations

Production issues

• Existing root cause: pass its issue_id and fresh cited_items to update_issue. Citations accumulate; inactive issues resurface, and ignored, monitored, or merged issues reject new evidence.
• Monitoring: select or create a Judge with continuous online evaluation above 0% sampling, then call update_issue with user_status=monitored, judge_id, and the exact behavior_id from its production version. If the transfer fails, retry with the same IDs.
• Conflicts: updates are atomic. On a conflict, reread get_issue before retrying; after an uncertain response, check the timeline before resending a message.

create_issue returns { issue: { id, issue_key } }; a duplicate issue_key conflicts. update_issue returns { issue }; read events through get_issue.

Attribute mappers

Attribute mappers copy incoming span attributes to public Judgment attribute keys during ingestion.

Agent memory

These tools operate on shared project memory.

The old names are no longer available. Update list_agent_memory_entries to list_memory_entries, fetch_agent_memory_files to read_memory_files, search_agent_memory_files to search_memory_files, and write_memory to write_memory_file. Inputs, outputs, and stored memory are unchanged.

Documentation bridge

These two read-only tools are registered on the authenticated data MCP for convenience, so coding agents can consult current product guidance without a second integration. Both proxy the docs site’s own hosted MCP server at docs.judgmentlabs.ai/mcp (its search_docs and get_page tools), so their inputs and results match that server exactly.

Verify the live schema

If a field or tool differs from this page, inspect the tool list exposed to the current connection. Available tools can vary by authorization scope and account configuration. Reconnect after an authorization-scope change so the client refreshes tools/list.