Edit and publish a judge
Save an Agent Judge revision as a draft, verify it, and promote the version you want to use in production.
Save an Agent Judge revision as a draft when you want to test changes before using them in production. Publishing or promoting a version makes it the production version; saving a draft does not.
Prerequisites
- A saved Agent Judge and permission to edit it
- Representative traces or a dataset for checking the revised judge
This guide covers the platform’s Agent Judge editor. To change Python source, follow the Code Judge workflow.
Understand the version labels
| Label | Meaning |
|---|---|
| Selected version | The version you are currently inspecting or using as the starting point for edits. |
| latest | The most recent saved version. It can be a draft. |
| production | The version designated for production use. It can be older than the latest version. |
Selecting a version does not promote it. A version can carry both the latest and production badges.
Save and verify a draft
- Open Judges, choose the judge, and select the version you want to revise.
- Edit the Rubric or Settings. For monitoring changes, check the online evaluation settings.
- Save > Create draft using the proposed minor version for backward-compatible changes, or the major version for incompatible output or behavior changes.
- Run an offline test with the draft version explicitly selected. Compare it with the previous version on the same dataset, and inspect disagreements before publishing.
Changes to the judge’s shared description use a plain Save action and do not create a new version. Changes to its evaluation configuration do.
Publish the version
- Saved draft: Select it in the version selector and confirm Promote to Production.
- New edits: Choose Save > Publish to production.
The published version has the production badge. For online evaluation, confirm its Online monitoring is enabled with sampling above zero, then check a new trace’s result.
Publishing does not change earlier test runs.
Return to an earlier version
Select the earlier saved version and confirm Promote to Production. Its monitoring settings become active too.
A newer draft can remain latest while the earlier version serves production.
Troubleshooting
- If you cannot switch versions, save or Discard your unsaved changes first.
- If production still uses the old settings, check whether you saved a draft without promoting it.
- If a test uses the wrong judge revision, explicitly select the intended version in the test configuration before starting another run.
Next step
Calibrate the judge to turn disagreements into evidence for the next revision.