Skip to Content
Judgment Labs
DocumentationTracingIntegrations

OpenTelemetry instrumentation libraries

Send spans from FastAPI, SQLAlchemy, Celery and other OpenTelemetry instrumentation packages to Judgment.

Judgment didn't write these packages. They all connect the same way, so this one page covers them.

Configure Judgment credentials in the process that runs the application before registering an instrumentor. After a real application request or service call, verify received spans.

Give the instrumentor Judgment's provider

Tracer.init doesn't install a global OpenTelemetry provider. An instrumentor that isn't handed a provider uses the global one, which by default is a no-op, so its spans go nowhere. A plain FastAPIInstrumentor.instrument_app(app) after Tracer.init exports nothing.

Give the instrumentor Judgment's provider in one of three ways, in this order of preference:

  1. Register it with Judgment. Tracer.registerOTELInstrumentation(X()) calls X().instrument(tracer_provider=<Judgment's provider>) for you.
  2. Pass the provider yourself when the library's API instruments one object at a time: tracer_provider=JudgmentTracerProvider.get_instance().
  3. Install Judgment globally with JudgmentTracerProvider.install_as_global_tracer_provider() when you can't reach the call. Only the first global install wins; see Frameworks with built-in OpenTelemetry.

Install the dependencies for the FastAPI example in your application's Python environment:

python -m pip install judgeval fastapi httpx opentelemetry-instrumentation-fastapi opentelemetry-instrumentation-httpx

Then register the HTTPX instrumentor and pass Judgment's provider to FastAPI:

app.py
from fastapi import FastAPI
from judgeval import JudgmentTracerProvider, Tracer
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor

tracer = Tracer.init(project_name="lunch-order-agent")
Tracer.registerOTELInstrumentation(HTTPXClientInstrumentor())

app = FastAPI()
FastAPIInstrumentor.instrument_app(
    app, tracer_provider=JudgmentTracerProvider.get_instance()
)

@app.middleware("http")
async def activate_tracer(request, call_next):
    tracer.set_active()
    return await call_next(request)

Web servers: activate the tracer per request

Judgment's provider sends each span to the tracer that's active in the current execution context. A web server may run requests in contexts that never saw Tracer.init. The judgeval repository's own FastAPI example handles this with the middleware shown above: keep the object Tracer.init returns and call tracer.set_active() at the start of each request. In Flask, do the same in a @app.before_request function.

Library table

Install judgeval, the library, and the package in the second column. "Register" means Tracer.registerOTELInstrumentation(Class()).

LibraryPackageClassHow to wire it
FastAPIopentelemetry-instrumentation-fastapiFastAPIInstrumentorRegister before creating FastAPI(), or FastAPIInstrumentor.instrument_app(app, tracer_provider=...)
Starletteopentelemetry-instrumentation-starletteStarletteInstrumentorStarletteInstrumentor.instrument_app(app, tracer_provider=...)
Flaskopentelemetry-instrumentation-flaskFlaskInstrumentorRegister before creating Flask(__name__), or FlaskInstrumentor().instrument_app(app, tracer_provider=...)
Djangoopentelemetry-instrumentation-djangoDjangoInstrumentorRegister in manage.py and in wsgi.py/asgi.py, before Django loads settings
aiohttp serveropentelemetry-instrumentation-aiohttp-serverAioHttpServerInstrumentorRegister
Any ASGI appopentelemetry-instrumentation-asgiOpenTelemetryMiddlewareapp = OpenTelemetryMiddleware(app, tracer_provider=...)
Any WSGI appopentelemetry-instrumentation-wsgiOpenTelemetryMiddlewareapp = OpenTelemetryMiddleware(app, tracer_provider=...)
httpxopentelemetry-instrumentation-httpxHTTPXClientInstrumentorRegister
requestsopentelemetry-instrumentation-requestsRequestsInstrumentorRegister
aiohttp clientopentelemetry-instrumentation-aiohttp-clientAioHttpClientInstrumentorRegister
SQLAlchemyopentelemetry-instrumentation-sqlalchemySQLAlchemyInstrumentorRegister before create_engine, or SQLAlchemyInstrumentor().instrument(engine=engine, tracer_provider=...)
psycopg 3opentelemetry-instrumentation-psycopgPsycopgInstrumentorRegister
asyncpgopentelemetry-instrumentation-asyncpgAsyncPGInstrumentorRegister
PyMongoopentelemetry-instrumentation-pymongoPymongoInstrumentorRegister
MySQL Connectoropentelemetry-instrumentation-mysqlMySQLInstrumentorRegister
sqlite3opentelemetry-instrumentation-sqlite3SQLite3InstrumentorRegister
Redisopentelemetry-instrumentation-redisRedisInstrumentorRegister
Celeryopentelemetry-instrumentation-celeryCeleryInstrumentorCall Tracer.init and register inside a worker_process_init signal handler, so each worker process has its own tracer
AWS Lambdaopentelemetry-instrumentation-aws-lambdaAwsLambdaInstrumentorRegister at module level and call Tracer.force_flush() before the handler returns. See Tracing patterns

Every tracer_provider=... above is JudgmentTracerProvider.get_instance(), imported with from judgeval import JudgmentTracerProvider. Module paths follow the package name: opentelemetry-instrumentation-httpx is from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor.

Celery, as a full example. Start a reachable Redis broker and a worker consuming the task queue, and replace the example broker URL with your own. Enqueue and complete a task to generate spans:

worker.py
from celery import Celery
from celery.signals import worker_process_init
from judgeval import Tracer
from opentelemetry.instrumentation.celery import CeleryInstrumentor

app = Celery("lunch_orders", broker="redis://localhost:6379/0")

@worker_process_init.connect(weak=False)
def init_tracing(*args, **kwargs):
    Tracer.init(project_name="lunch-order-agent")
    Tracer.registerOTELInstrumentation(CeleryInstrumentor())

LLM instrumentation families

Two projects publish instrumentors for model providers and agent frameworks.

FamilyPackagesHow it reaches JudgmentMapping in Judgment
OpenInference (Arize)openinference-instrumentation-<name>, module openinference.instrumentation.<name>RegisterBuilt-in OpenInference mapper: span kind, input, output, model, tokens
OpenLLMetry (Traceloop)opentelemetry-instrumentation-<name> (Python), @traceloop/instrumentation-<name> (TypeScript)RegisterBuilt-in GenAI mapper: model, provider, tokens. Prompts written as indexed keys may not fill input and output

If a span arrives with the right model and tokens but empty input or output, write a mapper for the library's keys. See Attribute keys and mappers.

On this page