OpenTelemetry instrumentation libraries

Send spans from FastAPI, SQLAlchemy, Celery and other OpenTelemetry instrumentation packages to Judgment.

Judgment didn’t write these packages. They all connect the same way, so this one page covers them.

Give the instrumentor Judgment’s provider

Tracer.init doesn’t install a global OpenTelemetry provider. An instrumentor that isn’t handed a provider uses the global one, which by default is a no-op, so its spans go nowhere. A plain FastAPIInstrumentor.instrument_app(app) after Tracer.init exports nothing.

Give the instrumentor Judgment’s provider in one of three ways, in this order of preference:

1. Register it with Judgment. Tracer.registerOTELInstrumentation(X()) calls X().instrument(tracer_provider=<Judgment's provider>) for you.
2. Pass the provider yourself when the library’s API instruments one object at a time: tracer_provider=JudgmentTracerProvider.get_instance().
3. Install Judgment globally with JudgmentTracerProvider.install_as_global_tracer_provider() when you can’t reach the call. Only the first global install wins; see Frameworks with built-in OpenTelemetry.

from fastapi import FastAPI
from judgeval import JudgmentTracerProvider, Tracer
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor

tracer = Tracer.init(project_name="lunch-order-agent")
Tracer.registerOTELInstrumentation(HTTPXClientInstrumentor())

app = FastAPI()
FastAPIInstrumentor.instrument_app(
    app, tracer_provider=JudgmentTracerProvider.get_instance()
)

@app.middleware("http")
async def activate_tracer(request, call_next):
    tracer.set_active()
    return await call_next(request)

import { HttpInstrumentation } from "@opentelemetry/instrumentation-http";
import { JudgmentTracerProvider, Tracer } from "judgeval";

JudgmentTracerProvider.getInstance();
Tracer.registerOTELInstrumentation(new HttpInstrumentation());

export const tracerReady = Tracer.init({ projectName: "lunch-order-agent" });

Web servers: activate the tracer per request

Judgment’s provider sends each span to the tracer that’s active in the current execution context. A web server may run requests in contexts that never saw Tracer.init. The judgeval repository’s own FastAPI example handles this with the middleware shown above: keep the object Tracer.init returns and call tracer.set_active() at the start of each request. In Flask, do the same in a @app.before_request function.

Library table

Install judgeval, the library, and the package in the second column. “Register” means Tracer.registerOTELInstrumentation(Class()).

Every tracer_provider=... above is JudgmentTracerProvider.get_instance(), imported with from judgeval import JudgmentTracerProvider. Module paths follow the package name: opentelemetry-instrumentation-httpx is from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor.

Celery, as a full example:

from celery import Celery
from celery.signals import worker_process_init
from judgeval import Tracer
from opentelemetry.instrumentation.celery import CeleryInstrumentor

app = Celery("lunch_orders", broker="redis://localhost:6379/0")

@worker_process_init.connect(weak=False)
def init_tracing(*args, **kwargs):
    Tracer.init(project_name="lunch-order-agent")
    Tracer.registerOTELInstrumentation(CeleryInstrumentor())

LLM instrumentation families

Three projects publish instrumentors for model providers and agent frameworks.

OpenLIT has a judgeval helper that passes Judgment’s tracer to openlit.init and turns off OpenLIT metrics:

from judgeval import Tracer
from judgeval.integrations.openlit import Openlit

Tracer.init(project_name="lunch-order-agent")
Openlit.initialize()

Import it from judgeval.integrations.openlit; from judgeval.integrations import Openlit raises ImportError. Extra keyword arguments go straight to openlit.init().

If a span arrives with the right model and tokens but empty input or output, write a mapper for the library’s keys. See Attribute keys and mappers.