Skip to content
Judgment Labs
Esc
↑↓navigate↵open⌘Jpreview
On this page

Frameworks with built-in OpenTelemetry

Send spans from Python frameworks that already emit OpenTelemetry, like Google ADK and Pipecat, to Judgment.

Google ADK, Pipecat, LiveKit Agents, and Pydantic AI already create OpenTelemetry spans. They send them to the global OpenTelemetry tracer provider unless you hand them a provider directly.

Tracer.init does not set the global provider. It builds a private one so it can’t clash with other tracing in your app. For these frameworks, add one call right after init:

from judgeval import JudgmentTracerProvider, Tracer

Tracer.init(project_name="lunch-order-agent")
JudgmentTracerProvider.install_as_global_tracer_provider()

OpenTelemetry lets only the first caller set the global provider. If something else set it first (another vendor’s SDK, a framework’s own setup_tracing()), the call logs a warning and returns False, and the framework’s spans go to that other provider. Check the return value once when you set this up.

All four frameworks on this page are Python. The TypeScript SDK has the same call, JudgmentTracerProvider.installAsGlobalTracerProvider(). For a TypeScript framework:

  • If it emits OpenTelemetry spans through the global provider, call JudgmentTracerProvider.installAsGlobalTracerProvider() after await Tracer.init().
  • If it ships its own exporter instead, send OTLP directly.

Google ADK

ADK traces agent runs, model calls, and tool calls through the global provider. The judgeval repository’s ADK example also sets two OpenTelemetry environment variables before importing ADK so message content is captured on the spans:

import os

os.environ["OTEL_SEMCONV_STABILITY_OPT_IN"] = "gen_ai_latest_experimental"
os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "SPAN_AND_EVENT"

from judgeval import JudgmentTracerProvider, Tracer

Tracer.init(project_name="lunch-order-agent")
JudgmentTracerProvider.install_as_global_tracer_provider()

from google.adk.runners import Runner  # noqa: E402
from google.adk.sessions import InMemorySessionService  # noqa: E402

Build your Agent and Runner as usual after these lines (pip install judgeval google-adk). To group runs into Judgment sessions, call Tracer.set_session_id(...) from an ADK plugin’s before_run_callback, as that example does. See Add context.

Pipecat

Pipecat traces each conversation, turn, and STT, LLM, and TTS service call. Install the global provider before you build the pipeline, then turn tracing on for the worker:

from judgeval import JudgmentTracerProvider, Tracer
from pipecat.pipeline.pipeline import Pipeline
from pipecat.pipeline.worker import PipelineParams, PipelineWorker

Tracer.init(project_name="lunch-order-agent")
JudgmentTracerProvider.install_as_global_tracer_provider()

def build_worker(pipeline: Pipeline, session_id: str) -> PipelineWorker:
    return PipelineWorker(
        pipeline,
        params=PipelineParams(enable_metrics=True),
        enable_tracing=True,
        conversation_id=session_id,
    )
  • enable_metrics=True turns on Pipecat’s metrics, which adds time-to-first-byte to service spans.
  • Don’t call Pipecat’s setup_tracing(). It installs its own global provider, and whichever runs first wins.
  • PipelineTask still works but Pipecat deprecated it in 1.3.0 in favor of PipelineWorker.

Install with pip install judgeval pipecat-ai.

LiveKit Agents

The framework package is livekit-agents, not livekit (that’s the realtime client).

pip install judgeval livekit-agents
from judgeval import JudgmentTracerProvider, Tracer
from livekit.agents.telemetry import set_tracer_provider

Tracer.init(project_name="lunch-order-agent")
set_tracer_provider(JudgmentTracerProvider.get_instance())

set_tracer_provider hands the provider to LiveKit only, so you don’t need the global install. LiveKit runs each job in its own process, so keep these lines at module level in the file that defines your entrypoint; they run again in every job process.

Pydantic AI

Pydantic AI emits OpenTelemetry GenAI spans (gen_ai.input.messages, gen_ai.output.messages, gen_ai.request.model, gen_ai.usage.*). Judgment’s GenAI mapper reads those keys for input, output, model, and tokens. Pass Judgment’s provider to Pydantic AI:

pip install judgeval pydantic-ai
from judgeval import JudgmentTracerProvider, Tracer
from pydantic_ai import Agent
from pydantic_ai.models.instrumented import InstrumentationSettings

Tracer.init(project_name="lunch-order-agent")
Agent.instrument_all(
    InstrumentationSettings(tracer_provider=JudgmentTracerProvider.get_instance())
)

agent = Agent("openai:gpt-5.2", instructions="Help the user order lunch for their office.")
result = agent.run_sync("order me mediterranean food to feed 30 hungry engineers")
print(result.output)
Tracer.force_flush()

Don’t add OpenInference’s OpenInferenceSpanProcessor through span_processors=. It rewrites attributes when a span ends, but Judgment’s processor runs first and copies the attributes before that, so the OpenInference attributes never reach Judgment.

Was this page helpful?