Authentication and projects
How ingest and personal API keys, organization IDs and project selectors authenticate and scope each request.
Every request to Judgment authenticates with a Judgment API key, then scopes the work to one organization and one project.
Credentials
| Value | What it is | Where to get it |
|---|---|---|
API key (JUDGMENT_API_KEY) |
An ingest API key to send traces, or a personal API key that acts as you. | Ingest: Settings > Ingest API Keys. Personal: Settings > Personal API Keys. |
Organization ID (JUDGMENT_ORG_ID) |
Picks one organization you belong to. An ingest key already belongs to one organization, which the server uses regardless of this value. The SDKs still read JUDGMENT_ORG_ID, so set it for ingest keys too. |
Copy organization ID in the organization menu or the settings sidebar |
| Project name | Picks a project in the SDK. | The name you pass to Tracer.init or Judgeval |
| Project ID | Picks a project over OTLP, HTTP, and MCP headers. | Copy project ID in the project menu, or the Project ID column on the Projects page |
Choose an API key
Judgment has two kinds of API key. For sending traces, use an ingest API key. Personal API keys also work for sending traces, but an ingest key is safer to put in an agent’s environment.
| Ingest API key (recommended for tracing) | Personal API key | |
|---|---|---|
| Belongs to | Your organization | You |
| Created by | Organization admins, in Settings > Ingest API Keys | Any member, in Settings > Personal API Keys |
| What it can do | Send traces and resolve a project by name, with the Developer role. Every other request, including Tracer.tag, is rejected with 403. |
Everything your role allows through the SDK, HTTP API, and MCP |
| If the creator leaves the organization | Keeps working | Stops working for that organization |
| Value format | Starts with jli_ |
No prefix |
| Use it for | Production agents, CI, and any service that only sends traces | Local development, datasets, tests, SQL, Tracer.tag, and the MCP API-key fallback |
Because an ingest key cannot read your data, a leaked one exposes much less than a leaked personal key. It can still send traces and create projects in your organization, so store it as a secret and rotate it if it is exposed. Store both keys as secrets in the server or worker environment. Don’t publish them in source, logs, browser bundles, or public examples.
Ingest API keys
Open Settings > Ingest API Keys. Every member can see the masked keys; only organization admins can create, rotate, or delete them. Each key has a name that is unique within the organization, such as production or ci.
| Action | Effect on existing credentials |
|---|---|
| Create Ingest API Key | Adds a separate named key. Existing keys stay active. |
| Rotate Ingest API Key | Replaces the selected key’s value and immediately invalidates its old value. Other keys stay active. |
| Delete | Invalidates the selected key. Requests still using it stop sending traces. |
- Select Create Ingest API Key, enter a name, and select Create.
- In New API Key Created, use the copy button and store the value in your secret manager before selecting Close. The full value is shown only once.
- Set it as
JUDGMENT_API_KEYfor the service that sends traces. Send a trace and check that it reaches the expected project.
There is no overlap period when you rotate a key. To change a key without interrupting traffic, create a second named key, move the consumers to it, and then delete the old key.
Personal API keys
Open Settings > Personal API Keys to manage your own keys. A personal key acts as you, with your role in each organization. Keys have no scopes; a key’s name helps you identify a consumer, such as staging, ci, or laptop, but doesn’t limit what it can reach.
| Action | Effect |
|---|---|
| Create API Key | Adds a named key. Other keys keep working. The full value is shown once; the copy button copies JUDGMENT_API_KEY=<key>. |
| Rotate API Key | Replaces the selected key’s value. The old value stops working right away. Other keys keep working. |
| Delete API Key | Removes the selected key. The key marked default can’t be deleted, but it can be rotated. |
The key table shows masked values. If you lose a key, rotate it or create a new one.
Headers
| Client | Authentication | Organization | Project |
|---|---|---|---|
| Python and TypeScript SDK | JUDGMENT_API_KEY |
JUDGMENT_ORG_ID |
Project name in code |
OTLP (/otel/v1/traces) |
Authorization: Bearer <key> |
X-Organization-Id (personal keys; ignored for ingest keys) |
X-Project-Id (required) |
HTTP API (/v1/...) |
Authorization: Bearer <personal key> |
X-Organization-Id |
Project ID in the path |
MCP (https://mcp.judgmentlabs.ai) |
Authorization: Bearer <OAuth token or personal key> |
X-Organization-Id (optional) |
X-Project-Id (optional) |
MCP does not accept ingest API keys. MCP also supports OAuth 2.1 with PKCE in clients that can open a browser. See Connect your coding agent and MCP tools.
How a project is chosen
| Path | Selector | What happens |
|---|---|---|
| SDK | Project name | The SDK resolves the name to an ID at startup. If no project has that name, Judgment creates it. Creating a project needs the Developer role or higher. |
| SDK with a Viewer key | Project name | The SDK can’t resolve or create the project, logs a warning, and turns trace export off. Your app keeps running. |
| OTLP | X-Project-Id |
Use the ID from Copy project ID for a project in the key’s organization (ingest key) or the organization you send in X-Organization-Id (personal key). |
| MCP | project_id argument or X-Project-Id |
A bound connection fills in the project for you. |
A typo in the project name creates a second project. If traces don’t show up where you expect, check the Projects page for a near-duplicate name.
Use one project per agent. Separate development, staging, and production with environments inside that project, set from the deployment.environment.name resource attribute. Traces without an environment show as No environment.
Roles
Roles are set per organization: Owner, Admin, Developer, Viewer. There are no project-level roles. A personal key can do what its user’s role allows: Viewer reads, Developer writes and creates projects, Admin deletes datasets and test configs and manages members. An ingest key acts with the Developer role, limited to sending traces and resolving projects. See Members and roles.
Status codes
| Status | Cause |
|---|---|
400 |
OTLP request without X-Project-Id; a body that can’t be decompressed or decoded |
401 |
Missing or malformed Authorization header; invalid key; missing X-Organization-Id on a personal-key request |
402 |
The organization is over its usage limit (Billing and plans) |
403 |
You are not a member of the organization, or the organization ID is malformed |
403 |
Ingest API keys can only be used to send traces and resolve projects: an ingest key was used for another request. Use a personal API key for it. |
403 |
The organization runs in your own cloud (BYOC). Hosted Judgment only serves billing, usage, and members for these organizations. |
404 |
SDK or HTTP SQL for a project that doesn’t exist in the organization (Query with SQL) |
413 |
A span’s attributes exceed 8 MiB (Ingest contract) |
429 |
Rate limited (Rate limits) |
503 |
Authentication or membership lookup is temporarily unavailable |
Rate limits
Judgment rate-limits a few paths per organization with token buckets. Each bucket has a burst capacity and a refill rate. A request that finds a token goes through; one that finds the bucket empty is limited.
The values below are the defaults Judgment ships with. Limits are set per organization, so yours may differ. To raise a limit, contact support.
Limits
| Scope | Covers | Burst | Refill | One token is |
|---|---|---|---|---|
| Online evaluations | Judges running on incoming traces and sessions | 100 | 100 per minute | One evaluation run (one judge on one trace or session) |
| Test runs | Evaluations created by test runs | 100 | 100 per minute | One evaluation run (examples × judges) |
| SDK and HTTP SQL | client.sql() and POST /v1/projects/{projectId}/sql |
40 | 4 per second | One request |
One API call that fans out into many evaluation runs can use many tokens at once.
At the limit
| Path | What happens |
|---|---|
| Online evaluations | Evaluations over the limit are skipped, not queued. |
| Evaluation API calls | 429 with a Retry-After header and {"error": "Rate limit exceeded"} |
| SDK and HTTP SQL | 429 with a Retry-After header and {"error": "QUERY_RATE_LIMITED", "hint": ...}. 503 if the limiter is unavailable. |
| Trace ingestion | No request rate limit. |
Usage limits
Rate limits are separate from your plan’s usage. When an organization is over its Judgment Unit (JU) limit, trace ingestion returns 402 with {"error": "Usage limit exceeded"}. Remaining JUs don’t lift a rate limit, and a higher rate limit doesn’t add JUs. See Billing and plans.