Members and roles
What each role can do in a Judgment organization, and how to invite members and change roles.
Roles in Judgment are set per organization. A member has one of four roles, and that role applies in every project in the organization. There are no project-level roles.
There are two kinds of API key:
| Key | Belongs to | What it can do |
|---|---|---|
| Personal API key | The person who created it | Acts with that person’s role, so a Viewer’s key can only read. |
| Ingest API key | The organization, not a person | Acts as a Developer and can only send traces. |
Only Admins and Owners can create and manage ingest API keys, under Settings > Ingest API Keys.
What each role can do
| Task | Viewer | Developer | Admin | Owner |
|---|---|---|---|---|
| See traces, Issues, judges, datasets, and test runs | Yes | Yes | Yes | Yes |
| Chat with the Judgment Agent and run SQL | Yes | Yes | Yes | Yes |
| Send traces with the SDK (it resolves or creates the project) | No | Yes | Yes | Yes |
| Create and edit judges (including their actions), datasets, test runs, and Issues | No | Yes | Yes | Yes |
| Create projects | No | Yes | Yes | Yes |
| Create Scheduled Tasks and manage your own | No | Yes | Yes | Yes |
| Manage everyone’s Scheduled Tasks | No | No | Yes | Yes |
| Delete datasets and test configs in the app | No | No | Yes | Yes |
| Delete datasets and test configs through the SDK or API | No | Yes | Yes | Yes |
| Delete projects | No | No | Yes | Yes |
| Create and manage ingest API keys | No | No | Yes | Yes |
| Complete the Tracing step in onboarding | No | No | Yes | Yes |
| Connect or disconnect Slack | No | No | Yes | Yes |
| Change billing, usage limits, and usage alerts | No | No | Yes | Yes |
| Invite and remove Developers and Viewers | No | No | Yes | Yes |
| Change a member’s role to Admin | No | No | No | Yes |
A Viewer’s key can still send traces straight to the OTLP endpoint, which doesn’t check roles. The SDK path needs Developer because it looks up the project by name first. See Authentication and projects.
Only the owner of a Scheduled Task can run it. See Scheduled Tasks.
Who can set up tracing
Developers can instrument from code by following Set up tracing. Only Admins and Owners can complete the in-app Tracing step.
Invite a member
Admins and Owners can invite people.
- Open Settings, then Members.
- Select Invite members.
- Enter the email address and choose a role. The form only offers roles below your own, so an Admin sees Developer and Viewer, and an Owner also sees Admin.
- Select Invite.
The person has to accept the invitation while signed in with the same email address it was sent to. Invitations don’t expire. Inviting the same email again updates the role on the pending invitation. Inviting someone who is already a member fails.
Change a role
Open the member’s row menu and select Change role. In Edit Member Role, pick the new role and select Update.
The new role has to be below yours, as with invitations.
Remove a member or an invitation
- For a member, open the row menu and select Delete Member. You can only remove people whose role is below yours.
- For a pending invitation, select Delete Invitation. An Admin can’t delete a pending Admin invitation.
Removing a member takes away their access to this organization. Their Judgment account and their API keys still exist, but the keys stop working for this organization.
Things to know
- Viewers don’t see other Viewers in the member list.
- Single sign-on and MFA requirements are set per email domain by Judgment. contact support to set them up.