--- title: "Cookbook: Root-cause an Issue" description: "Take one Issue from the inbox to a shipped fix and a judge that watches for it." sidebar: label: "Cookbook: Root-cause an Issue" seo: title: "Cookbook: Root-cause an Issue | Judgment Docs" description: "Take one Issue from the inbox to a shipped fix and a judge that watches for it." --- Your agent has been in production for a week. Nobody has complained and the dashboards are green, but you suspect it's failing somewhere you can't see. ## Which Issue should I look at first? Open Issues. The inbox shows **Needs review** Issues, sorted by **Severity**. Near the top is a **High** severity Issue, trend **First seen**, titled "Menu lookups time out and the agent silently drops the restaurant". Read its three short sections before anything else. On this Issue they'd say something like: - **Description**: `get_menu` returns `504 Gateway Timeout` for one restaurant, the agent retries once, then drops the restaurant from the shortlist without telling the user. - **Impact**: requests for larger catering orders end up with fewer options; the user never learns a restaurant was skipped. - **Consider**: tell the user when a restaurant couldn't be checked, or retry with a longer timeout before dropping it. ## What actually happened in the trace? Under **Evidence**, each row is one trace, with what happened, where, and when. Open one. It's the request from the [example trace](/documentation/start/example-agent#one-trace), "order me mediterranean food to feed 30 hungry engineers", and the failure is in the middle: - tool `get_menu` x3: Olive & Thyme and Falafel House return menus, Cedar Grill fails (`504 Gateway Timeout`) - tool `get_menu` retry: Cedar Grill fails again - llm call: drop Cedar Grill and continue with the other two The final response recommends Olive & Thyme, $612 total, delivered by 12:15 PM, and never mentions Cedar Grill. The response looks fine, which is why nobody noticed: the failure only shows up mid-trajectory. Open a second evidence trace and make sure it fails the same way. If it fails for a different reason, say so in a comment under **Activity**; the diagnosis may be grouping two problems. ## How big is it? Before you prioritize a fix, get numbers. Select the Issue in the inbox and choose **Ask Judgment**: ```text For the Issue "Menu lookups time out and the agent silently drops the restaurant": over the last 7 days, how many traces had a get_menu call that returned a 504? Which restaurants were affected? In how many of those traces did the final response mention that a restaurant was skipped? Cite traces. ``` The agent searches your traces with Judgment's virtual SQL (vsql), reads the spans it needs, and answers with citations you can click. You might learn the timeouts all come from two restaurants, or that they spike at 11:30 AM when everyone orders. Either one would change the fix. The same question works from Codex, Claude Code or Cursor with the Judgment MCP server connected. The agent uses `get_issue` to read the Issue and `sql_query` to count. See [Connect your coding agent](/documentation/connect/connect-your-coding-agent). ## Who fixes it, and how? Choose **Fix with** on the Issue and pick your coding agent. The prompt includes the diagnosis and cited traces; see [Triage and fix](/documentation/issues/triage-and-fix#hand-it-to-your-coding-agent) for exactly what it contains. For this Issue, a reasonable fix has two parts: retry `get_menu` with backoff before giving up, and when a restaurant still can't be checked, say so in the response ("I couldn't load Cedar Grill's menu, so I left it out"). Your coding agent proposes the change, and you review and ship it like any other pull request. Judgment doesn't merge or deploy anything. ## How do I make sure it doesn't come back? Two things, and both are manual today. ### Keep the failing cases Open an evidence trace and use **Add to Dataset** to save it to a dataset of regression examples, or ask the agent to create a dataset and mention the Issue with `@` ("Turn recent failures into test cases"). Run your agent over that dataset before and after the fix and compare the runs. [Run a test from your code](/documentation/tests/run-a-test-from-your-code) shows how. ### Watch production with a judge Back on the Issue, choose **Review**, then **Monitor with Judge**. This opens a Judgment Agent chat that creates a judge for this failure, or reuses one you have. You can steer it: ```text Create a trace-level judge: "Did a tool fail to return data the agent needed, and did the agent tell the user?" Labels: No tool failure, Failed and told user, Failed silently. Turn on online monitoring for production. ``` The Issue becomes **Monitored** once the agent links that judge and the **Failed silently** label to it, which needs the judge running on production traffic ([Triage and fix](/documentation/issues/triage-and-fix#review-monitor-it-or-dismiss-it) has the exact condition). You can then see the rate of silent failures per day, filter traces by that label, and send it to Slack with a [judge action](/documentation/judges/judge-actions). ## How do I know the fix worked? After the fix ships, the judge's label rate for **Failed silently** should drop toward zero on new traffic. See [Where labels live](/documentation/judges/overview#where-labels-live). A Monitored Issue doesn't go back to **Regressed** ([what happens to an Issue](/documentation/issues/overview#what-happens-to-an-issue)), so the judge is the signal. ## Do it from your coding agent With the Judgment MCP server connected, paste this into Codex, Claude Code or Cursor: ```text Using Judgment, list the Needs review issues in the lunch-order-agent project and pick the highest-severity one. Read the issue and two of its cited traces, tell me whether the diagnosis holds, and if the cause is in this repo, propose the smallest fix. Don't change the issue's status until I confirm. ``` **[Triage and fix](/documentation/issues/triage-and-fix)** Every inbox and review action in one place. **[Cookbook: Testing](/documentation/tests/cookbook-testing)** Turn those regression examples into a CI gate.