Tracing patterns

Keep one run as one complete trace when your agent streams, queues work, runs serverless, or delegates.

The basic setup assumes one function call is one agent run. Find the section that matches how your agent runs.

Use one trace for an uninterrupted unit of agent work. Keep the trace open while the application can take the next step by itself. End it when the application must wait for a user, request, queue delivery, schedule, human approval, or another external signal, and continue in a new trace within the same session.

Every completed root becomes a trace that Judgment can score. Leave plumbing that contains no agent work out of the trace: health checks, status polls, storage helpers, and routine middleware. HTTP auto-instrumentation can create a root for every route. If the application already uses it, open a named agent span around the agent work and put the application input, output, and session ID on that span. Span kinds and names are listed in Trace conventions.

Streaming responses

The handler returns before generation finishes, so the root span has to follow the stream, not the handler. Ending the root in the handler closes the trace too early. Buffering the whole stream changes the app’s behavior.

In Python, observe the async generator itself, not the handler that returns StreamingResponse. The root stays active while the framework consumes the generator. disable_generator_yield_span=True skips a child span per chunk.

from fastapi.responses import StreamingResponse
from judgeval import Tracer

@Tracer.observe(span_type="agent", span_name="lunch_order.stream", disable_generator_yield_span=True)
async def stream_reply(chat_id: str, messages: list[dict]):
    Tracer.set_session_id(chat_id)
    async for chunk in generate_chunks(messages):
        yield chunk

def make_response(chat_id: str, messages: list[dict]) -> StreamingResponse:
    return StreamingResponse(stream_reply(chat_id, messages))

For callback-based streams, such as the Vercel AI SDK in TypeScript:

1. Pass Tracer.getOTELTracer() when the framework’s telemetry setup accepts a tracer. Enabling telemetry without it can drop the framework spans.
2. Start the root with Tracer.getOTELTracer().startActiveSpan(...) before generation, and set the session ID inside it. Don’t use Tracer.startActiveSpan here: it ends the span when the callback (or the promise it returns) finishes, which is before the stream is done.
3. Return the stream as soon as generation starts.
4. End the root from the framework’s finish, error, and abort callbacks. Wait until the final output is known, until save steps and other callbacks end, and until all model and tool spans end.
5. Record the final output on the root from the finish callback.
6. Mark an aborted stream as an error and save a short, useful part of the output.
7. Flush after the root ends if the runtime can freeze (see Serverless and Cloudflare Workers).

Tracer.set_session_id() and Tracer.set_customer_id() in Python, or Tracer.setSessionId() and Tracer.setCustomerId() in TypeScript, write to the active span. Call them while the streaming root is active. The Vercel AI SDK page has a full sample.

Two shapes fail quietly:

• Buffering. Reading the whole stream before responding produces a clean trace and a broken app, because the user loses the live reply.
• The floating root. Returning the stream from outside the traced callback, with the traced promise left un-awaited, drops the root span silently. The framework’s own spans then export as sessionless roots, and the session ID lands nowhere. The platform still shows traces, so the loss is easy to miss. Check for the named root even when traffic appears.

To verify, test a full stream and an aborted stream. The full stream should save the full output on your named root. The aborted stream should mark that root as an error. If your named root is missing while framework spans appear, you have the floating-root shape.

Queues, workflows, and durable work

A root span can’t stay open across a queue, a checkpoint, a process stop, or an approval wait. Make each autonomous part its own trace and give every part the same session ID, such as the job or workflow ID.

Trace the action that starts work

A job passes through many hands between submission and completion, and most of them only move it along. Trace a handler when the work itself starts there, and skip it when the job just passes through.

A request or command that drops the job on a queue and returns is a pass-through. The work starts later, in the worker, and the worker’s trace is the first one in the session.

A handler that starts the run in its own process, or restarts one after a crash, is different. The work begins there, so that handler gets its own root in the job’s session.

The process boundary decides this, and the return time doesn’t matter. A handler that starts a background loop and returns immediately still starts the work in its own process. Record the job input on it and end it before the job waits.

Long-running loops

Each iteration or step can be one durable part. Keep every part of the run in the same session. End and flush the current part before saving its checkpoint, even when the loop runs inside a long-running server. An unclean kill then loses at most the in-flight part, never finished work sitting in the export queue.

Queues and worker processes

Pass the job or session ID in the task message or saved state. The worker starts a root when it receives the job and sets that ID as the session.

from judgeval import Tracer

task = {"job_id": job_id, "payload": payload}
queue.send(task)


@Tracer.observe(span_type="agent", span_name="job.run")
def process_task(task: dict):
    Tracer.set_session_id(task["job_id"])
    return run(task["payload"])

An automatic retry remains part of the same job. Each attempt is its own trace in the same session, with the attempt number as an attribute.

Whether the failed attempt leaves a trace depends on how the worker died:

• A graceful stop with time to flush exports the failed attempt as an error trace.
• A hard kill loses the unfinished root, so the retry may be the only trace for that step.

Neither SDK installs signal handlers. A graceful stop only exports the attempt if your own shutdown code makes the observed function raise and then calls force_flush().

Workflow engines (Temporal and similar)

Workflow code replays, so it can’t own a live app root. Trace the activities instead:

• One trace per activity execution, created by the activity itself.
• Every trace carries the workflow or job ID as its session ID.
• Record the attempt number as an attribute so retries are searchable.
• Initialize the tracer at import time in each process that creates spans: the producer and every worker.
• Mark handled failures on the span that returned them. A cancelled or timed-out attempt is a failure: mark it as an error, not a clean trace with empty output.
• End and flush the activity root in the activity’s outer finally block before the activity returns.

from judgeval import Tracer
from temporalio import activity

Tracer.init(project_name="lunch-order-agent", environment="production")

@Tracer.observe(span_type="agent", span_name="place_order")
async def _place_order(req: OrderRequest) -> OrderResult:
    info = activity.info()
    Tracer.set_session_id(info.workflow_id)
    Tracer.set_attribute("lunch.attempt", info.attempt)
    return await submit_order(req)

@activity.defn
async def place_order(req: OrderRequest) -> OrderResult:
    try:
        return await _place_order(req)
    finally:
        Tracer.force_flush()

Temporal’s own TracingInterceptor can attach later activities under an already-ended submit trace; add it only if you need its transport spans.

Human approval

End the current trace before the app waits. The approval request or signal gets a root in the same job session only when its handler does app work, such as resuming the run in its own process. Record fields you will search for, such as the approver and phase, as attributes. If the handler only records the approval, the next autonomous part starts the new root. Approval status polls don’t start work, so they don’t need spans.

To verify, kill a worker mid-activity, let the engine retry, and resume the workflow. The activity traces should stay in the same session, and the retry should have a higher attempt number.

Serverless and Cloudflare Workers

A serverless runtime can freeze right after it responds, dropping spans still waiting to export. Initialize once per runtime instance (module scope, not inside the handler), flush after the root ends and before returning, and shut down only when the process stops, not after each request:

Tracer.init(project_name="lunch-order-agent", environment="production")

def lambda_handler(event, context):
    result = handle_request(event["chat_id"], event["message"])  # observed root
    Tracer.force_flush()
    return {"statusCode": 200, "body": result}

In TypeScript, await Tracer.forceFlush() after the root ends.

TypeScript bundlers

A bundled runtime can give each entry point its own copy of judgeval or @opentelemetry/api. The copy you initialized is then not the copy your handler uses, so nothing records. Keep one server-side copy of both packages. For Next.js, list both in serverExternalPackages:

export default {
  serverExternalPackages: ["judgeval", "@opentelemetry/api"],
};

Load tracing from the runtime’s instrumentation or bootstrap path before you import traced model clients. Log once at initialization; a second log when a handler loads points to a second copy.

Cloudflare Workers

Import from judgeval/workers. Workers don’t read process.env, so pass apiKey, organizationId, and apiUrl explicitly, and prefer projectId over projectName to skip the lookup. Unlike the Node tracer, this Tracer.init throws if the project can’t be resolved. The tracer uses AsyncLocalStorage from async_hooks, so enable the nodejs_compat compatibility flag in your Wrangler config.

The credentials only arrive with each request’s env, so start Tracer.init on the first request and reuse the same promise for every later request in that isolate. If initialization fails, clear the promise so the next request tries again.

import { Tracer } from "judgeval/workers";

let tracerReady: Promise<Tracer> | undefined;

function initTracer(env: Env): Promise<Tracer> {
  tracerReady ??= Tracer.init({
    apiKey: env.JUDGMENT_API_KEY,
    organizationId: env.JUDGMENT_ORG_ID,
    apiUrl: "https://api.judgmentlabs.ai",
    projectId: env.JUDGMENT_PROJECT_ID,
    environment: "production",
  }).catch((error) => {
    tracerReady = undefined;
    throw error;
  });
  return tracerReady;
}

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    await initTracer(env);
    const reply = await handleRequest(await request.text()); // your observed agent
    ctx.waitUntil(Tracer.forceFlush());
    return new Response(reply);
  },
};

To verify, deploy and export a full trace on a cold call, meaning the first request after the runtime spins up. The init log should appear once, and the completed trace should arrive before the runtime freezes. Silent export loss is the failure mode here: the app works while the platform shows nothing.

Model calls through a CLI or subprocess

No wrapper can see inside a subprocess. If the app calls a provider SDK directly, use the provider’s integration instead. Otherwise, record each subprocess invocation as an llm child:

1. Keep the app root open through the call, parsing, and result saving.
2. Parse the result and call recordLLMMetadata() with model, provider, non_cached_input_tokens, output_tokens, cache_read_input_tokens, cache_creation_input_tokens, and total_cost_usd.
3. Set cost only when the subprocess reports it. Recording 0 for an unknown cost makes a paid session look free and makes the total unreliable.
4. Mark a non-zero exit, error payload, or denial as an error, even when the process exits cleanly.
5. Record useful values such as duration, time to first token, turn count, and stop reason as separate attributes so they can be queried without opening the raw result.

from judgeval import Tracer
from opentelemetry.trace import Status, StatusCode

@Tracer.observe(span_type="llm", span_name="cli.model")
def call_model(prompt: str) -> dict:
    completed = run_cli(prompt)
    parsed = parse_cli_output(completed.stdout)

    metadata = {
        "model": parsed.model,
        "provider": parsed.provider,
        "non_cached_input_tokens": parsed.input_tokens,
        "output_tokens": parsed.output_tokens,
    }
    if parsed.cost_usd is not None:
        metadata["total_cost_usd"] = parsed.cost_usd

    Tracer.recordLLMMetadata(metadata)
    if completed.returncode != 0 or parsed.is_error:
        Tracer.get_current_span().set_status(
            Status(StatusCode.ERROR, completed.stderr or "CLI call failed")
        )

    return parsed.result

When the subprocess owns the conversation, use its conversation ID as the session ID. Keep the root active until you parse and set that ID. Save your app’s request or wrapper ID as a separate attribute. When the app resumes the conversation, pass the same subprocess ID back to the CLI and use it again as the session ID.

For Claude Code structured output, read model IDs from the keys in modelUsage; one call may use more than one model. Check is_error, api_error_status, and permission_denials even when the exit code is zero.

To verify, run one real task and open the llm span. It should show the reported model, tokens, and cost. If the CLI reported a cost, the session total should not be $0.00.

Distributed services

One live request that crosses services stays one trace when you propagate OpenTelemetry context between them. Use one project for every service and give each a distinct service.name when you initialize its tracer, not on individual spans (it defaults to the project name; see Add context). One trace then shows every hop while the services stay distinguishable.

Set the session ID on the upstream root before injecting the outgoing headers. The SDK includes that ID in Judgment baggage, and continuing the trace restores it, so downstream child spans inherit the same session ID. Session, customer, and customer user IDs travel in baggage; the trace’s session still comes from the upstream root.

Inject the current trace context into the outgoing request:

import httpx
from judgeval.trace.propagation import inject

headers = {}
inject(headers)
httpx.post(MENU_SERVICE_URL, json=payload, headers=headers)

import { propagation } from "judgeval";

const headers: Record<string, string> = {};
propagation.inject(headers);
await fetch(MENU_SERVICE_URL, { method: "POST", headers, body });

Continue the trace before the downstream service starts any span. A span that starts first gets a new trace ID, which makes the request appear as two unrelated traces. Middleware is a reliable place to continue the trace because it covers every incoming request:

with Tracer.continue_trace(request.headers):
    result = get_menu(payload)

await Tracer.continueTrace(req.headers, async () => {
  return getMenu(payload);
});

Delayed queue work needs a stable job or session ID instead of the live request’s trace context. Start a new trace when the worker receives the task; see Queues, workflows, and durable work.

To verify, send one request through both services. The downstream spans should appear under the upstream root in one trace. Two traces mean the context didn’t arrive or was restored after a downstream span had already started.

Subagents

Keep ordinary delegated work inside the parent’s trace. Fork a linked trace only when a subagent performs a meaningful unit of work that should be inspected or evaluated on its own.

Don’t fork only because the implementation uses another class, process, or model. The product boundary is whether the delegated result stands on its own.

Set fork=True in Python or fork: true in TypeScript on the observed agent function:

from judgeval import Tracer

@Tracer.observe(span_type="agent", span_name="menu_research_subagent", fork=True)
def menu_research_subagent(task: str) -> str:
    return research(task)

const menuResearchSubagent = Tracer.observe(
  async function menuResearchSubagent(task: string): Promise<string> {
    return research(task);
  },
  { spanType: "agent", spanName: "menu_research_subagent", fork: true },
);

The parent trace keeps an invocation span, and the subagent runs as a new root linked by judgment.link.* attributes. A forked trace is a normal trace: online judges evaluate it like any other. Keep the delegated task as the forked trace input and the returned result as its output.

A fork only happens when a parent span is active and recording at call time. Called from detached background work with no active span, the function runs as an ordinary span or a new unlinked root. In Python, generator functions never fork. For lower-level control in Python, use Tracer.start_linked_trace(). If the subagent resumes after a queue or approval wait, also follow the durable work boundary.

Troubleshooting