--- title: "Set up Judgment with your coding agent" description: "Paste one prompt into your coding agent, and it connects Judgment, instruments your repo and shows you what Judgment found." sidebar: label: "Set up with your coding agent" seo: title: "Set up Judgment with your coding agent | Judgment Docs" description: "Paste one prompt into your coding agent, and it connects Judgment, instruments your repo and shows you what Judgment found." --- Your coding agent can set up Judgment in your repo from start to finish. Paste this prompt into Claude Code, Codex or Cursor from the root of the repo: ```text Set up Judgment in this repo by following https://docs.judgmentlabs.ai/documentation/start/agent-setup.md ``` You sign in once in the browser, and you can create your account there. Claude Code and Codex also need one restart to load the Judgment tools, and the agent tells you how. The agent does the rest: it creates a project and an ingest API key, adds tracing to your agent, sends sample traces, and reports the judges and Issues Judgment created from them. The rest of this page is the agent's instructions. ## Instructions for the coding agent Work through these steps in order. Tell the user what you're doing at each step in one line. Never print, log or commit an API key. ### 1. Connect the Judgment MCP server If you already have Judgment MCP tools such as `list_organizations`, skip to step 2. If the `judgment` server is already configured but its tools aren't available, don't add it again. In Codex, run `codex mcp login judgment`. Then go to the sign-in table below. Otherwise, add the server `https://mcp.judgmentlabs.ai` for the client you're running in: | Client | Command or config | | --- | --- | | Claude Code | `claude mcp add --transport http judgment https://mcp.judgmentlabs.ai` | | Codex | `codex mcp add judgment --url https://mcp.judgmentlabs.ai`, then `codex mcp login judgment` | | Cursor | Add `"judgment": { "url": "https://mcp.judgmentlabs.ai" }` under `mcpServers` in `.cursor/mcp.json` | In Claude Code and Codex, a server added during a session usually isn't available until a new session starts. Stop here until the user confirms. Ask them to sign in and restart as their client needs: | Client | What the user does | | --- | --- | | Claude Code in a terminal | Quit, then run `claude --continue` in this directory to keep the conversation. Run `/mcp`, select **judgment**, then select **Authenticate**. Say "done". | | Claude Code desktop app | Run `/mcp`. If **judgment** is listed, select it, then **Authenticate**, then say "done". Otherwise, start a new session in this directory, sign in through `/mcp`, and paste the continue prompt. | | Codex in a terminal | Complete the browser sign-in that `codex mcp login judgment` opened. Then quit, run `codex resume --last` in this directory, and say "done". | | Codex app | Complete the browser sign-in that `codex mcp login judgment` opened. Then start a new thread in this directory and paste the continue prompt. | | Cursor | Open the MCP settings, select **judgment**, then select **Sign in**. Say "done". | Codex runs `codex mcp login` outside its sandbox, so it asks the user to approve the command first. The continue prompt picks up where this session stopped: ```text Continue setting up Judgment by following https://docs.judgmentlabs.ai/documentation/start/agent-setup.md ``` Tell the user what to expect in the browser: - A new user can create an account with Google or GitHub, then name their organization before authorizing. - If the page asks for a project, leave **All Projects in This Organization** selected. A project-bound connection can't create projects or keys. After the user confirms, call `list_organizations`. If the tools still aren't available, ask the user to start a new session in this directory and paste the continue prompt. ### 2. Choose the organization and project 1. Call `list_organizations`. If there's more than one, ask the user which to use. 2. Call `list_projects` for that organization. If a project matches this repo's name, ask whether to reuse it. 3. Otherwise, call `create_project` with the repo's directory name. ### 3. Create an ingest API key If the repo's `.env` already sets `JUDGMENT_API_KEY` and `JUDGMENT_ORG_ID` for the organization from step 2, reuse them and skip to step 4. Each key you create stays active until someone deletes it. Otherwise, call `create_ingest_api_key` with the repo's directory name as `name`. If the name is taken, add a numeric suffix, such as `-2`. Write the key and the organization ID to the repo's `.env` file, creating it if needed: ```bash JUDGMENT_API_KEY= JUDGMENT_ORG_ID= ``` Before you write the key, run `git ls-files .env`. If it prints `.env`, the file is tracked by git, so stop and ask the user where to store the key. Otherwise, make sure `.gitignore` lists `.env`. The key is shown only once, so don't call the tool again to read it. ### 4. Add tracing An earlier session may have added part of the tracing. Check what's already there, keep it, and finish only what's missing. 1. Call `read_doc_page` for `/documentation/connect/set-up-tracing`, or fetch `https://docs.judgmentlabs.ai/documentation/connect/set-up-tracing.md`. 2. Find the agent's entry points and its model and tool calls. Install the judgeval SDK for the repo's language. 3. Initialize the tracer once, with the project name from step 2, and load credentials from `.env`. 4. Trace the agent's model, tool and agent calls as the tracing page describes. Keep the changes small and leave product behavior as it is. Show the user the diff before you run anything. ### 5. Send sample traces Run the instrumented agent about 10 times with varied, realistic requests. Include a few that should fail safely, such as an out-of-scope request. Use test data or mocks, never production data or real customers. Flush the tracer before the process exits. Then call `get_trace_health_summary` for the project. If a check fails, read its remediation, fix the instrumentation, and run the samples again. A pending check isn't a pass. ### 6. Show what Judgment found 1. Call `start_onboarding_analyses` for the project. Judgment creates judges suited to the agent and runs Issue detection over the sample traces. If it returns `started: false`, its `reason` says why. 2. Call `get_onboarding_status` every minute or so until neither run is `running` or `waiting`. Tell the user you're waiting on Judgment. 3. If a run's status is `failed`, report its `error_message` to the user and stop. 4. Call `list_judges` and `list_issues`, then summarize for the user: - the judges Judgment created and the labels each one applies - each Issue found, with its title and what it means for their agent - anything you couldn't verify, such as a failing health check Link to pages with `https://app.judgmentlabs.ai/org//project//`. Add `judges/` for a judge. End by telling the user that new traces get labeled as they arrive. ## If something goes wrong | Symptom | Fix | | --- | --- | | The MCP tools don't appear after adding the server | Restart in this directory with `claude --continue` or `codex resume --last`. In a desktop app, start a new session and paste the continue prompt. Then sign in as step 1 describes. | | A tool returns `403` | The user's role is too low. Creating keys and starting analyses need Admin. | | The SDK logs `401: Invalid API key` | A `JUDGMENT_API_KEY` or `JUDGMENT_ORG_ID` already set in the shell overrides `.env`. Unset it, or load `.env` with override turned on. | | `get_trace_health_summary` shows no traces | Check that `.env` loads before the tracer starts, and that the tracer is flushed before the process exits. | | `start_onboarding_analyses` returns `awaiting_traces` | No traces have arrived yet. Run step 5 again, then check `get_trace_health_summary`. | | `start_onboarding_analyses` returns `already_complete` | The project already finished setup. Read its judges and Issues with `list_judges` and `list_issues`. | | `get_onboarding_status` shows a `failed` run | Report the run's `error_message`, then [contact support](/documentation/start/get-help). | To set up a repo by hand instead, follow [Set up tracing](/documentation/connect/set-up-tracing).