OpenTelemetry instrumentation libraries
Send spans from FastAPI, SQLAlchemy, Celery and other OpenTelemetry instrumentation packages to Judgment.
Judgment didn't write these packages. They all connect the same way, so this one page covers them.
Configure Judgment credentials in the process that runs the application before registering an instrumentor. After a real application request or service call, verify received spans.
Give the instrumentor Judgment's provider
Tracer.init doesn't install a global OpenTelemetry provider. An instrumentor that isn't handed a provider uses the global one, which by default is a no-op, so its spans go nowhere. A plain FastAPIInstrumentor.instrument_app(app) after Tracer.init exports nothing.
Give the instrumentor Judgment's provider in one of three ways, in this order of preference:
- Register it with Judgment.
Tracer.registerOTELInstrumentation(X())callsX().instrument(tracer_provider=<Judgment's provider>)for you. - Pass the provider yourself when the library's API instruments one object at a time:
tracer_provider=JudgmentTracerProvider.get_instance(). - Install Judgment globally with
JudgmentTracerProvider.install_as_global_tracer_provider()when you can't reach the call. Only the first global install wins; see Frameworks with built-in OpenTelemetry.
Install the dependencies for the FastAPI example in your application's Python environment:
python -m pip install judgeval fastapi httpx opentelemetry-instrumentation-fastapi opentelemetry-instrumentation-httpxThen register the HTTPX instrumentor and pass Judgment's provider to FastAPI:
from fastapi import FastAPI
from judgeval import JudgmentTracerProvider, Tracer
from opentelemetry.instrumentation.fastapi import FastAPIInstrumentor
from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor
tracer = Tracer.init(project_name="lunch-order-agent")
Tracer.registerOTELInstrumentation(HTTPXClientInstrumentor())
app = FastAPI()
FastAPIInstrumentor.instrument_app(
app, tracer_provider=JudgmentTracerProvider.get_instance()
)
@app.middleware("http")
async def activate_tracer(request, call_next):
tracer.set_active()
return await call_next(request)Web servers: activate the tracer per request
Judgment's provider sends each span to the tracer that's active in the current execution context. A web server may run requests in contexts that never saw Tracer.init. The judgeval repository's own FastAPI example handles this with the middleware shown above: keep the object Tracer.init returns and call tracer.set_active() at the start of each request. In Flask, do the same in a @app.before_request function.
Library table
Install judgeval, the library, and the package in the second column. "Register" means Tracer.registerOTELInstrumentation(Class()).
| Library | Package | Class | How to wire it |
|---|---|---|---|
| FastAPI | opentelemetry-instrumentation-fastapi | FastAPIInstrumentor | Register before creating FastAPI(), or FastAPIInstrumentor.instrument_app(app, tracer_provider=...) |
| Starlette | opentelemetry-instrumentation-starlette | StarletteInstrumentor | StarletteInstrumentor.instrument_app(app, tracer_provider=...) |
| Flask | opentelemetry-instrumentation-flask | FlaskInstrumentor | Register before creating Flask(__name__), or FlaskInstrumentor().instrument_app(app, tracer_provider=...) |
| Django | opentelemetry-instrumentation-django | DjangoInstrumentor | Register in manage.py and in wsgi.py/asgi.py, before Django loads settings |
| aiohttp server | opentelemetry-instrumentation-aiohttp-server | AioHttpServerInstrumentor | Register |
| Any ASGI app | opentelemetry-instrumentation-asgi | OpenTelemetryMiddleware | app = OpenTelemetryMiddleware(app, tracer_provider=...) |
| Any WSGI app | opentelemetry-instrumentation-wsgi | OpenTelemetryMiddleware | app = OpenTelemetryMiddleware(app, tracer_provider=...) |
| httpx | opentelemetry-instrumentation-httpx | HTTPXClientInstrumentor | Register |
| requests | opentelemetry-instrumentation-requests | RequestsInstrumentor | Register |
| aiohttp client | opentelemetry-instrumentation-aiohttp-client | AioHttpClientInstrumentor | Register |
| SQLAlchemy | opentelemetry-instrumentation-sqlalchemy | SQLAlchemyInstrumentor | Register before create_engine, or SQLAlchemyInstrumentor().instrument(engine=engine, tracer_provider=...) |
| psycopg 3 | opentelemetry-instrumentation-psycopg | PsycopgInstrumentor | Register |
| asyncpg | opentelemetry-instrumentation-asyncpg | AsyncPGInstrumentor | Register |
| PyMongo | opentelemetry-instrumentation-pymongo | PymongoInstrumentor | Register |
| MySQL Connector | opentelemetry-instrumentation-mysql | MySQLInstrumentor | Register |
| sqlite3 | opentelemetry-instrumentation-sqlite3 | SQLite3Instrumentor | Register |
| Redis | opentelemetry-instrumentation-redis | RedisInstrumentor | Register |
| Celery | opentelemetry-instrumentation-celery | CeleryInstrumentor | Call Tracer.init and register inside a worker_process_init signal handler, so each worker process has its own tracer |
| AWS Lambda | opentelemetry-instrumentation-aws-lambda | AwsLambdaInstrumentor | Register at module level and call Tracer.force_flush() before the handler returns. See Tracing patterns |
Every tracer_provider=... above is JudgmentTracerProvider.get_instance(), imported with from judgeval import JudgmentTracerProvider. Module paths follow the package name: opentelemetry-instrumentation-httpx is from opentelemetry.instrumentation.httpx import HTTPXClientInstrumentor.
Celery, as a full example. Start a reachable Redis broker and a worker consuming the task queue, and replace the example broker URL with your own. Enqueue and complete a task to generate spans:
from celery import Celery
from celery.signals import worker_process_init
from judgeval import Tracer
from opentelemetry.instrumentation.celery import CeleryInstrumentor
app = Celery("lunch_orders", broker="redis://localhost:6379/0")
@worker_process_init.connect(weak=False)
def init_tracing(*args, **kwargs):
Tracer.init(project_name="lunch-order-agent")
Tracer.registerOTELInstrumentation(CeleryInstrumentor())LLM instrumentation families
Two projects publish instrumentors for model providers and agent frameworks.
| Family | Packages | How it reaches Judgment | Mapping in Judgment |
|---|---|---|---|
| OpenInference (Arize) | openinference-instrumentation-<name>, module openinference.instrumentation.<name> | Register | Built-in OpenInference mapper: span kind, input, output, model, tokens |
| OpenLLMetry (Traceloop) | opentelemetry-instrumentation-<name> (Python), @traceloop/instrumentation-<name> (TypeScript) | Register | Built-in GenAI mapper: model, provider, tokens. Prompts written as indexed keys may not fill input and output |
If a span arrives with the right model and tokens but empty input or output, write a mapper for the library's keys. See Attribute keys and mappers.