Set up tracing

Send your agent's first trace to Judgment from Python or TypeScript and check that it arrived intact.

Instrument one real agent path so its root, model calls, tool calls, input, output, and session reach the intended Judgment project. Start with supported instrumentation, then add manual spans only for work it doesn’t capture.

Prerequisites

• Install the SDK. Python 3.10+: pip install judgeval openai (judgeval 1.3.3+). Node 18+: npm install judgeval openai (judgeval 1.3.4+).
• Set JUDGMENT_API_KEY and JUDGMENT_ORG_ID as described in Authentication and projects. Use an ingest API key for JUDGMENT_API_KEY: it can only send traces, so it is safer than a personal API key. A personal API key also works if your role is Developer or higher. The SDK still reads JUDGMENT_ORG_ID with an ingest API key, so set both. Set JUDGMENT_API_URL only if Judgment gave you a different API URL (default https://api.judgmentlabs.ai).
• Choose an explicit project name and deployment environment. See How a project is chosen.
• Identify the code path that performs real agent work and what one trace should cover. See Glossary.

Use auto-instrumentation

Two integration families cover most applications:

Enable the integration that matches the application. Open one app root around the work so integration spans become its children, then add Tracer.observe only for missing work such as custom tools and retrievals.

TypeScript Tracer.wrap supports OpenAI clients only. Python Tracer.wrap supports OpenAI, Anthropic, Together, and Google GenAI (google.genai.Client). Tracer.init doesn’t install a global OpenTelemetry provider, so a third-party instrumentor exports nothing until you register it with Tracer.registerOTELInstrumentation(...). See OTel instrumentation libraries.

TypeScript OpenTelemetry auto-instrumentation requires all three of these:

1. A file the runtime loads before the application.
2. Instrumentations registered with Judgment’s tracer provider before the application loads the modules they patch.
3. A CommonJS build, because the instrumentations patch modules when they are required.

Tracer.registerOTELInstrumentation() needs Judgment’s tracer provider to exist. Tracer.init() creates it, but only after an await, and a CommonJS file can’t use top-level await. So create the provider with JudgmentTracerProvider.getInstance(), register the instrumentations, and then start Tracer.init():

import { JudgmentTracerProvider, Tracer } from "judgeval";
import { OpenAIInstrumentation } from "@opentelemetry/instrumentation-openai";

JudgmentTracerProvider.getInstance();
Tracer.registerOTELInstrumentation(new OpenAIInstrumentation());

export const tracerReady = Tracer.init({
  projectName: "lunch-order-agent",
  environment: "production",
});

Import this file first in the entrypoint, or preload it with the runtime’s require flag (node --require ./instrumentation.js). Spans that start before Tracer.init() resolves are not recorded, so await tracerReady before the application handles its first request. If Tracer.registerOTELInstrumentation() runs before the provider exists, the SDK logs No active tracer provider. Instrumentation was not registered. and the instrumentation exports nothing. For bundlers, also follow Serverless and Cloudflare Workers.

Initialize tracing first

Initialize once in every process that creates spans, before constructing wrapped clients, observed functions, workers, or the application.

from judgeval import Tracer

Tracer.init(
    project_name="lunch-order-agent",
    environment="production",
)

import { Tracer } from "judgeval";

await Tracer.init({
  projectName: "lunch-order-agent",
  environment: "production",
});

Use environment to distinguish development, staging, and production traces within a project. Traces without an environment show as No environment.

Tracer.init looks up the project by name and creates it if no project has that name (developer role required), so a typo in the project name sends traces to a new, empty project. With a viewer-role key the lookup fails, and the SDK logs a warning and exports nothing. Neither case raises an exception. Log the selected project and deployment environment at startup, but never log the API key.

To turn export off, skip Tracer.init, or leave JUDGMENT_API_KEY, JUDGMENT_ORG_ID, or the project name unset. The SDK logs a warning, and observed functions still run, untraced.

Trace the agent path

This example uses the lunch-order agent and assumes no framework integration. It creates one root, traces one tool, and wraps one supported model client. Keep only the pieces your selected integration doesn’t already provide. Set OPENAI_API_KEY before running it.

Every span is named explicitly. Production bundlers can strip function names, which otherwise leaves blank names in the trace view.

import tracer  # initializes tracing before the client is created

from judgeval import Tracer
from openai import OpenAI

client = Tracer.wrap(OpenAI())

@Tracer.observe(span_type="tool", span_name="search_nearby_restaurants")
def search_nearby_restaurants(cuisine: str) -> list[str]:
    return ["Olive & Thyme", "Falafel House", "Cedar Grill"]

@Tracer.observe(span_type="agent", span_name="lunch_order")
def handle_request(session_id: str, message: str) -> str:
    Tracer.set_session_id(session_id)
    restaurants = search_nearby_restaurants("mediterranean")
    response = client.chat.completions.create(
        model="gpt-5.2",
        messages=[
            {"role": "system", "content": f"Restaurants nearby: {restaurants}"},
            {"role": "user", "content": message},
        ],
    )
    return response.choices[0].message.content

if __name__ == "__main__":
    print(handle_request("chat-123", "order me mediterranean food to feed 30 hungry engineers"))
    Tracer.shutdown()

import "./tracer";

import { Tracer } from "judgeval";
import OpenAI from "openai";

const client = Tracer.wrap(new OpenAI());

const searchNearbyRestaurants = Tracer.observe(
  async function searchNearbyRestaurants(cuisine: string): Promise<string[]> {
    return ["Olive & Thyme", "Falafel House", "Cedar Grill"];
  },
  { spanType: "tool", spanName: "search_nearby_restaurants" },
);

const handleRequest = Tracer.observe(
  async function handleRequest(sessionId: string, message: string): Promise<string> {
    Tracer.setSessionId(sessionId);
    const restaurants = await searchNearbyRestaurants("mediterranean");
    const response = await client.chat.completions.create({
      model: "gpt-5.2",
      messages: [
        { role: "system", content: `Restaurants nearby: ${restaurants.join(", ")}` },
        { role: "user", content: message },
      ],
    });
    return response.choices[0]?.message.content ?? "";
  },
  { spanType: "agent", spanName: "lunch_order" },
);

console.log(await handleRequest("chat-123", "order me mediterranean food to feed 30 hungry engineers"));
await Tracer.shutdown();

Activate the root span

A span becomes the parent of new work only while it is active:

In TypeScript, Tracer.startSpan returns a span without activating it. Tracer.setSessionId() then has no active span to update, and later work doesn’t become a child. Raw OpenTelemetry context.with and trace.setSpan also don’t activate the SDK’s own context. Use an active boundary from the table.

Python has the same lifecycle distinction: Tracer.start_span() returns a span that you must manage. Prefer an active boundary whenever work should inherit that span.

Long-lived background work started while a request span is active can inherit the request forever. Start the background loop outside any active span and give each unit its own root.

Flush completed roots

Judgment starts evaluation when the completed root arrives, so every child must be sent before the root. End the root with a real duration, output, and status before flushing. Spans wait in a batch queue for about five seconds; anything still queued when a process stops is lost.

result = run_job(job)  # the observed root ends when this returns
Tracer.force_flush()

const result = await runJob(job);
await Tracer.forceFlush();

Verify the first trace

1. Run real agent work with tracing initialized, and read the startup logs. project_name not provided, api_key not provided, organization_id not provided, or Project '...' not found means nothing was exported.
2. Confirm the trace reached the expected project and environment. Storage is asynchronous, so give it a moment.
3. Open the trace and confirm the named app root contains the expected model and tool spans, input, output, and session ID.
4. Read the trace health results. Judgment checks root structure, nesting, timing, input/output, LLM metadata, and session ID consistency. A check with no status is still pending, not passing. See Trace health.
5. Confirm plumbing didn’t create extra traces and each request produced one expected trace.
6. Test one hard case: restart, stream abort, worker retry, approval resume, handled tool failure, or serverless cold call.

A coding agent can read the same health results with get_trace_health, get_trace_health_summary, and list_trace_health_failures through the Judgment MCP server.

Troubleshooting

Next step

Add context once the trace structure is healthy. Tracing patterns covers streaming, queues, serverless, and distributed services. The Tracer SDK Reference contains the complete Python and TypeScript APIs.