Skip to content
Judgment Labs
Esc
↑↓navigate↵open⌘Jpreview
On this page

Ingest contract (OTLP)

The OTLP/HTTP endpoint, headers, encodings, limits and status codes for trace ingest.

Judgment ingests traces over OTLP/HTTP. The judgeval SDKs use this endpoint, and any OpenTelemetry exporter can send to it directly. For a setup walkthrough, see Send OTLP directly.

Endpoint

Method URL Use
POST https://api.judgmentlabs.ai/otel/v1/traces Trace ingest.
POST https://api.judgmentlabs.ai/otel/v1/offline-traces Legacy SDK URL. Same behavior, except every span’s environment is forced to testing. New integrations should not use it.

The SDKs read the base URL from JUDGMENT_API_URL (default https://api.judgmentlabs.ai) and append /otel/v1/traces.

Headers

Header Required Value
Authorization Yes Bearer <JUDGMENT_API_KEY>. We recommend an ingest API key from Settings > Ingest API Keys. A personal API key from Settings > Personal API Keys also works.
X-Organization-Id Personal keys only Organization ID. The key’s user must be a member. An ingest key already belongs to one organization, so the server ignores this header for it.
X-Project-Id Yes Project ID. Names are not accepted.
Content-Type No A value containing application/json is parsed as OTLP/JSON. Any other value, or no header, is parsed as OTLP/protobuf.
Content-Encoding No gzip is supported (case-insensitive, may appear in a list such as gzip, identity).

An ingest API key can only send traces and resolve projects, and acts with the Developer role. A personal key authenticates as its user, and ingest has no role check, so a viewer’s personal key can send spans to this endpoint. The SDKs also resolve the project name to an ID first, which needs the Developer role. See Authentication and projects.

Project routing

Set X-Project-Id to the ID of the project the traces belong to.

Encodings

Encoding Request Success response
OTLP/protobuf ExportTraceServiceRequest, any non-JSON Content-Type 200, application/x-protobuf ExportTraceServiceResponse with no partial success
OTLP/JSON resourceSpans JSON, Content-Type containing application/json 200, {"status": "success"}

Limits

Limit Value Result when exceeded
Request body, as sent 50 MB Rejected before parsing. An uncompressed body can be up to 50 MB.
Gzip body after decompression 20 MB 400 {"error": "Failed to decompress request body"}. Applies only to gzip bodies.
Attributes per span 8 MiB 413 for the whole request. Nothing in the request is stored.

The per-span budget counts the serialized bytes of the span’s attributes, its resource attributes, its event attributes, and its link attributes. The 413 message names the oversized span and its largest attribute, so you can trim that attribute at the source. Large prompts and raw tool results are the usual cause.

Ingest has no request rate limit. It stops accepting traces when the organization reaches its usage limit (402). See Rate limits and Billing and plans.

Status codes

Status Body Cause
200 See Encodings Accepted.
400 {"error": "X-Project-Id header is required"} Missing X-Project-Id.
400 {"error": "Failed to decompress request body"} Invalid gzip, or more than 20 MB after decompression.
400 {"error": "Failed to decode trace request body"} Body is not valid OTLP for the detected encoding.
401 {"error": "Authentication failed", "message": ...} Missing or malformed Authorization, invalid key, or missing X-Organization-Id with a personal key.
402 {"error": "Usage limit exceeded", "message": ...} Organization is over its usage limit.
403 {"error": "Forbidden", "message": ...} Key’s user is not a member of the organization, malformed organization ID, or the organization is a hosted BYOC organization.
413 {"error": "Payload too large", "message": ...} A span is over the 8 MiB attribute budget.
503 {"error": "Service unavailable", "message": ...} API key or membership lookup is temporarily unavailable. Retry.

A 200 means the spans were accepted for storage. Storage is asynchronous, so a trace can take a short time to appear in Traces and SQL.

Resource attributes

Resource attribute Meaning
deployment.environment.name Environment of every span in the resource. deployment.environment is read as a fallback. An empty or missing value shows as No environment. There is no environment header.
service.name Service that emitted the spans. The SDKs default it to the project name.

Traces, roots, and sessions

Term Rule
Root span A span with no parent span ID.
Finalized span A span whose end time is after its start time (duration > 0). The SDKs also send partial spans whose end time equals the start time while work is in progress. Partial spans are stored but never finalize.
Finalized root A root span with duration > 0. Each finalized root triggers trace processing once, which is when online judges evaluate the trace.
Session Read only from the finalized root’s judgment.session_id, and only when it is a non-empty string. Session values on child spans do not assign the trace to a session.

Consequences:

  • A trace whose root never reaches Judgment, or arrives with zero duration, is stored but never evaluated. A common cause is an upstream service that propagates traceparent but does not export to Judgment, so the local root has a parent ID that never arrives.
  • Judges evaluate the trace as it is shortly after its finalized root arrives. Send child spans before or with the root. Spans that arrive later are stored, but judges don’t see them.

Attributes handled at ingest

Attribute Handling
judgment.span_name Replaces the stored span name, then is removed.
judgment.span_kind Stored as the span kind. Defaults to span when absent.
judgment.pending_trace_eval Read on finalized spans, attached to the root’s evaluation, then removed.

Attribute mappers run on every span before storage. See Attribute keys and mappers.

Was this page helpful?