Instrument your agent

Initialize Judgment tracing, capture one real agent path, export it safely, and verify the trace.

Instrument one real agent path so its root, model calls, tool calls, input, output, and session reach the intended Judgment project. Start with supported instrumentation, then add manual spans only for work it does not capture.

Prerequisites

• Install the Judgeval SDK for your language.
• Configure JUDGMENT_API_KEY and JUDGMENT_ORG_ID as described in Authentication.
• Choose an explicit project name for this environment. See Project routing.
• Identify the code path that performs real agent work and what one trace should cover. See the tracing data model.

Use auto-instrumentation

Two integration families cover most applications:

Enable the integration that matches the application. Open one app root around the work so integration spans become its children, then add Tracer.observe only for missing work such as custom tools and retrievals.

TypeScript OpenTelemetry auto-instrumentation requires all three of these:

1. A file the runtime loads before the application.
2. Instrumentations registered on Tracer before Tracer.init().
3. A CommonJS build, because the instrumentations patch modules when they are required.

import { Tracer } from "judgeval";
import { OpenAIInstrumentation } from "@opentelemetry/instrumentation-openai";

Tracer.registerOTELInstrumentation(new OpenAIInstrumentation());

await Tracer.init({ projectName: process.env.JUDGMENT_PROJECT_NAME });

Import this file first in the entrypoint, or preload it with the runtime’s require or import flag. For bundlers, also follow Bundled and serverless runtimes.

Initialize tracing first

Initialize once in every process that creates spans, before constructing wrapped clients, observed functions, workers, or the application.

import os
from judgeval import Tracer

Tracer.init(project_name=os.environ["JUDGMENT_PROJECT_NAME"])

import { Tracer } from "judgeval";

await Tracer.init({
  projectName: process.env.JUDGMENT_PROJECT_NAME,
});

Do not silently guess a fallback project. A missing project name can leave monitoring disabled, while a guessed name can send traces somewhere nobody is watching. Log the selected project and deployment environment at startup, but never log the API key.

Tracing is enabled when JUDGMENT_MONITORING is absent. Set JUDGMENT_MONITORING=false only when you intentionally need the deployment to stop exporting traces; do not require a separate true value that can be forgotten in production.

Trace the agent path

This example assumes no framework integration. It creates one root, traces one tool, and wraps one supported model client. Keep only the pieces your selected integration does not already provide.

Every span is named explicitly. Production bundlers can strip function names, which otherwise leaves blank names in the trace view.

import tracer  # initializes tracing before the client is created

from judgeval import Tracer
from openai import OpenAI

openai = Tracer.wrap(OpenAI())

@Tracer.observe(span_type="tool", span_name="retrieve_context")
def retrieve_context(question: str) -> str:
    return f"Reference material relevant to: {question}"

@Tracer.observe(span_type="agent", span_name="run_agent")
def run_agent(question: str) -> str:
    Tracer.set_session_id("example-session")
    context = retrieve_context(question)
    response = openai.chat.completions.create(
        model="gpt-5.6-luna",
        messages=[{
            "role": "user",
            "content": f"Question: {question}\n\nContext: {context}",
        }],
    )
    return response.choices[0].message.content

if __name__ == "__main__":
    print(run_agent("What is the capital of the United States?"))
    Tracer.shutdown()

import "./tracer";

import { Tracer } from "judgeval";
import OpenAI from "openai";

const openai = Tracer.wrap(new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
}));

const retrieveContext = Tracer.observe(async function retrieveContext(
  question: string,
): Promise<string> {
  return `Reference material relevant to: ${question}`;
}, { spanType: "tool", spanName: "retrieve_context" });

const runAgent = Tracer.observe(async function runAgent(
  question: string,
): Promise<string> {
  Tracer.setSessionId("example-session");
  const context = await retrieveContext(question);
  const response = await openai.chat.completions.create({
    model: "gpt-5.6-luna",
    messages: [{
      role: "user",
      content: `Question: ${question}\n\nContext: ${context}`,
    }],
  });
  return response.choices[0]?.message.content || "No answer";
}, { spanType: "agent", spanName: "run_agent" });

await runAgent("What is the capital of the United States?");
await Tracer.shutdown();

Activate the root span

A span becomes the parent of new work only while it is active:

In TypeScript, Tracer.startSpan returns a span without activating it. Tracer.setSessionId() then has no active span to update, and later work does not become a child. Raw OpenTelemetry context.with and trace.setSpan also do not activate the SDK’s own context. Use an active boundary from the table.

Python has the same lifecycle distinction: Tracer.start_span() returns a span that you must manage. Prefer an active boundary whenever work should inherit that span.

Long-lived background work started while a request span is active can inherit the request forever. Start the background loop outside any active span and give each unit its own root.

Flush completed roots

Judgment starts evaluation when the completed root arrives, so every child must be sent before the root. End the root with a real duration, output, and status before flushing. Spans wait in a batch queue for about five seconds; anything still queued when a process stops is lost.

result = run_job(job)  # the observed root ends when this returns
Tracer.force_flush()

const result = await runJob(job);
await Tracer.forceFlush();

Verify before you claim success

1. Run real agent work with monitoring enabled.
2. Confirm the trace reached the expected project and environment.
3. Open the trace and confirm the named app root contains the expected model and tool spans, input, output, and session ID.
4. Read the trace health results. Judgment checks root structure, nesting, timing, input/output, and LLM metadata. A check with no status is still pending, not passing.
5. Confirm plumbing did not create extra traces and each request produced one expected trace.
6. Test one hard case: restart, stream abort, worker retry, approval resume, handled tool failure, or serverless cold call.

A coding agent can read the same health results with get_trace_health, get_trace_health_summary, and list_trace_health_failures through the Judgment MCP server.

Troubleshooting

Next step

Add attributes and context once the trace structure is healthy. The Tracer SDK Reference contains the complete Python and TypeScript APIs.