MCP tools

Every tool on the Judgment MCP server, with its scope, minimum role and arguments.

The Judgment MCP server gives coding agents and other MCP clients read and write access to one organization’s projects. To set it up in Codex, Claude Code, or Cursor, see Connect your coding agent.

Endpoint

Item	Value
URL	`https://mcp.judgmentlabs.ai`
Transport	Streamable HTTP, stateless. `POST` only; other methods return `405`.
Auth	`Authorization: Bearer <token>`. The token is an OAuth access token or a Judgment API key. A request without a bearer token gets an OAuth challenge.
OAuth	OAuth 2.1 with PKCE (S256), issued by `https://auth.judgmentlabs.ai`.
Health	`GET /health`

Headers

Header	Required	Value
`Authorization`	Yes	`Bearer <oauth-token-or-api-key>`
`X-Organization-Id`	No	Organization ID. On an API-key connection, binds the connection to one organization, so tools don’t need `organization_id`. An OAuth connection is already bound to the organization you chose when you authorized.
`X-Project-Id`	No	Project ID (UUID). Binds the connection to one project. On an API-key connection, send it with `X-Organization-Id`.

A malformed X-Project-Id returns 400. X-Project-Id without X-Organization-Id on an API-key connection returns 400. On an OAuth connection, an X-Organization-Id or X-Project-Id that differs from the authorized organization or project returns 400.

Scope and roles

Each tool has a scope and a minimum organization role.

Scope	Meaning
Account	Works for any signed-in user.
Organization	Takes `organization_id`, unless the connection is bound to an organization.
Project	Takes `organization_id` and `project_id`, unless the connection is bound. Bound fields are removed from the tool’s input schema and filled in by the server.

Roles from lowest to highest are Viewer, Developer, Admin, and Owner. Viewers can call read tools. Writes need Developer. Deleting an automation, dataset, or test config needs Admin. On an OAuth connection, tools above your role are not listed. See Members and roles.

A connection bound to one project (by X-Project-Id or a project-scoped OAuth grant) hides organization and account tools, except list_organizations, discover_schema, search_docs, and read_doc_page.

Some API and SQL names, such as the behavior_id argument, still say behavior; it means label.

Tools

The arguments column lists each tool’s own inputs; ? marks an optional argument. Organization-scoped tools also accept organization_id?, and project-scoped tools accept organization_id? and project_id?; these are not repeated in the tables. evaluate_traces and evaluate_all_traces are enabled per organization; contact support to turn them on. They are listed only on a connection bound to that organization.

New judges are categorical: create_judge takes judgeType prompt (Agent judge), custom (Code judge), or deterministic, and at least two outputs (exactly two for deterministic, match first, then miss). Use get_judge_label and update_judge_label for labels; get_judge_result, get_behavior_detail, update_judge_result, and update_behavior still work as deprecated aliases.

Organizations

Tool	Scope	Minimum role	Arguments	What it does
`list_organizations`	Account	Any user	`limit?`, `offset?`	Organizations you belong to.

Projects

Tool	Scope	Minimum role	Arguments	What it does
`create_project`	Organization	Developer	`name`	Create a project.
`list_projects`	Organization	Viewer	none	Projects in the organization, with summary counts.

SQL

Tool	Scope	Minimum role	Arguments	What it does
`discover_schema`	Organization	Viewer	none	Markdown reference for the SQL tables, columns, examples, and limits. Call before `sql_query`.
`sql_query`	Project	Viewer	`sql`	Run one read-only `SELECT` against the project. Up to 1,000 rows and 5 MiB.

Trace health and evaluation

Tool	Scope	Minimum role	Arguments	What it does
`evaluate_all_traces`	Project	Developer	`judge_names?`, `limit?`	Queue judges to run on the most recent traces (default 100, max 1,000). Enabled per organization; contact support to turn it on.
`evaluate_traces`	Project	Developer	`trace_ids`, `judge_names?`	Queue judges to run on specific traces. Enabled per organization; contact support to turn it on.
`get_trace_health`	Project	Viewer	`trace_id`, `category?`	Check results for one trace.
`get_trace_health_summary`	Project	Viewer	`category?`, `start_time`, `end_time`	Check status and failure rates for a time range of up to 31 days.
`list_trace_health_failures`	Project	Viewer	`category?`, `check_id`, `start_time`, `end_time`, `limit?`, `cursor_trace_started_at?`, `cursor_trace_id?`	Traces failing one check in a time range.

Issues

Tool	Scope	Minimum role	Arguments	What it does
`create_issue`	Project	Developer	`issue_key?`, `title`, `diagnosis`, `impact_summary?`, `priority?`, `production_activity?`, `agent_reasoning?`, `cited_items`	Create an Issue backed by cited evidence. A duplicate `issue_key` returns a conflict.
`get_issue`	Project	Viewer	`issue_id`	One Issue with its cited evidence and event history.
`list_issues`	Project	Viewer	`user_status?`, `production_activity?`, `priority?`	Issues in the project, filterable by review status, activity, and priority.
`update_issue`	Project	Developer	`title?`, `diagnosis?`, `impact_summary?`, `priority?`, `production_activity?`, `agent_reasoning?`, `cited_items?`, `issue_id`, `user_status?`, `judge_id?`, `behavior_id?`, `message?`	Edit an Issue, comment on it, or change its review status (`needs_review`, `ignored`, `merged`, `monitored`). `resolved` is set by the system.

Judges

Tool	Scope	Minimum role	Arguments	What it does
`create_judge`	Project	Developer	`judge`	Create an Agent judge, a Code judge (uploaded bundle), or a deterministic judge, with its first production version. The judge type can’t change later.
`delete_judge`	Project	Developer	`judge_id`	Delete a judge and the labels that belong to it.
`get_judge`	Project	Viewer	`judge_id`, `include_code_url?`, `version?`	One version of a judge (latest by default), with its definition, outputs, and settings. For a Code judge, `include_code_url` returns a temporary download link for its bundle.
`list_judges`	Project	Viewer	`version?`	Judges in the project (latest version, or `version: "production"` for production judges and their enabled labels).
`update_judge`	Project	Developer	`judge_id`, `update`	Edit the description, or create a new version and optionally promote it to production.
`list_judge_action_groups`	Project	Viewer	`version?`	Action groups owned by judges, with group IDs and the judge version they belong to (`production` by default, or `latest`).
`set_judge_action_groups`	Project	Developer	`judge_id`, `source_version`, `target_version`, `promote_to_production`, `action_groups`	Replace a judge’s action groups by creating a new judge version. Pass the complete list; omitted groups are removed. Call `list_judge_action_groups` first.

Labels

Tool	Scope	Minimum role	Arguments	What it does
`get_judge_label`	Project	Viewer	`behavior_id`, `start_date?`, `end_date?`	Rate, last-applied time, and time series for one label on a production judge. Aliases: `get_judge_result`, `get_behavior_detail`.
`update_judge_label`	Project	Developer	`behavior_id`, `description?`	Edit a label description. Aliases: `update_judge_result`, `update_behavior`.

Automations

These tools manage project automations (rules). For alerts owned by a judge, use list_judge_action_groups and set_judge_action_groups.

Tool	Scope	Minimum role	Arguments	What it does
`create_automation`	Project	Developer	`name`, `description?`, `environment_names`, `conditions`, `combine_type`, `actions?`, `cooldown_period?`, `trigger_frequency?`	Create an automation. `environment_names` is required: `[]` means all environments, `[""]` means traces with no environment.
`delete_automation`	Project	Admin	`rule_id`	Delete an automation.
`get_automation`	Project	Viewer	`rule_id`	One automation.
`list_automations`	Project	Viewer	none	Automations in the project.
`update_automation`	Project	Developer	`rule_id`, `name?`, `description?`, `environment_names?`, `conditions?`, `combine_type?`, `actions?`, `active?`, `cooldown_period?`, `trigger_frequency?`	Edit an automation, or turn it on or off with `active`.

Datasets

Tool	Scope	Minimum role	Arguments	What it does
`add_examples_to_dataset`	Project	Developer	`dataset_id`, `examples`	Add examples. Creates a new dataset version.
`create_dataset`	Project	Developer	`name`, `schema`, `examples?`	Create a dataset with a JSON Schema and optional examples.
`delete_dataset`	Project	Admin	`dataset_id`	Delete a dataset. Also deletes its test configs and run metadata.
`get_dataset`	Project	Viewer	`dataset_id?`, `name?`, `version?`, `limit?`, `cursor_created_at?`, `cursor_example_id?`	One dataset (by ID or name) and a page of its examples.
`list_datasets`	Project	Viewer	none	Datasets with schema, example count, and current version.
`remove_examples_from_dataset`	Project	Developer	`dataset_id`, `example_ids`	Remove examples. Creates a new dataset version.

Tests

Tool	Scope	Minimum role	Arguments	What it does
`cancel_test_run`	Project	Developer	`test_run_id`	Cancel a pending or running test run.
`create_test_config`	Project	Developer	`name`, `description?`, `dataset_id`, `judge_ids`	Pair a dataset with judges.
`delete_test_config`	Project	Admin	`test_config_id`	Delete a test config and its run metadata.
`get_test_run`	Project	Viewer	`test_run_id`	Status and metadata for one test run.
`get_test_run_items`	Project	Viewer	`test_run_id`, `limit?`, `cursor?`	Per-example results and scores for one test run.
`list_test_configs`	Project	Viewer	`dataset_id?`	Test configs (a dataset paired with judges).
`list_test_runs`	Project	Viewer	`test_config_id?`, `dataset_id?`, `status?`, `limit?`	Test runs, newest first.
`run_test`	Project	Developer	`test_config_id`, `dataset_version_number?`, `judge_versions?`	Queue a test run that scores every example with every judge in the config.
`update_test_config`	Project	Developer	`test_config_id`, `name?`, `description?`	Rename a test config. The dataset and judges are fixed.

Agent Knowledge

Tool	Scope	Minimum role	Arguments	What it does
`list_memory_entries`	Project	Viewer	none	Paths of Agent Knowledge files and folders.
`read_memory_files`	Project	Viewer	`ids?`, `names?`	Read Agent Knowledge files by ID or path.
`search_memory_files`	Project	Viewer	`query`, `limit?`	Search Agent Knowledge files by path and body.
`write_memory_file`	Project	Developer	`name`, `body`	Create or update one Agent Knowledge file by path.

Attribute mappers

Tool	Scope	Minimum role	Arguments	What it does
`create_mapper`	Project	Developer	`name`, `rules`, `enabled?`, `override?`, `remove_source?`	Create an attribute mapper. Org-wide mappers need a Judgment admin.
`delete_mapper`	Project	Developer	`config_id`	Delete an attribute mapper.
`get_mapper`	Project	Viewer	`config_id`	One attribute mapper.
`list_mappers`	Project	Viewer	none	Attribute mappers for the project, including org-wide mappers.
`update_mapper`	Project	Developer	`config_id`, `name?`, `rules?`, `enabled?`, `override?`, `remove_source?`	Edit an attribute mapper.

Docs

Tool	Scope	Minimum role	Arguments	What it does
`read_doc_page`	Account	Any user	`route`	Read one Judgment docs page as Markdown.
`search_docs`	Account	Any user	`query`, `contentTypes?`, `filters?`, `limit?`	Search the Judgment docs.

Retired tools

These tools were removed in September 2026. Calling one returns Tool <name> not found.

Retired tool	Use instead
`search_traces`, `get_trace_detail`, `get_trace_spans`, `get_trace_span`, `read_span_attribute`, `get_trace_tags`, `get_trace_behaviors`	`sql_query` on `telemetry.traces`, `telemetry.spans`, `telemetry.scores`, `telemetry.trace_tags`
`search_sessions`, `get_session_detail`, `get_session_trace_ids`, `get_session_behaviors`, `get_session_trace_behaviors`	`sql_query`, grouping `telemetry.traces` by `session_id`
`get_test_run_graph`	`sql_query` on `evaluations.items`
`list_agent_memory_entries`, `fetch_agent_memory_files`, `search_agent_memory_files`, `write_memory`	`list_memory_entries`, `read_memory_files`, `search_memory_files`, `write_memory_file`
`write_issues`, `ignore_issue`, `reopen_issue`, `write_issue_comment`, `monitor_issue_with_judge`	`create_issue`, `update_issue`
`delete_judges`	`delete_judge`

For the SQL tables, see SQL schema.