Tracing Providers
OpenLLMetry Integration
Export OpenLLMetry traces to the Judgment platform.
OpenLLMetry integration sends traces from your OpenLLMetry-instrumented applications to Judgment. OpenLLMetry provides OpenTelemetry-based instrumentation for various LLM providers and frameworks.
Quickstart
Install Dependencies
uv add opentelemetry-instrumentation-openai judgeval openaipip install opentelemetry-instrumentation-openai judgeval openaiInitialize Integration
from judgeval import Tracer
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
Tracer.init(project_name="openllmetry_project")
Tracer.registerOTELInstrumentation(OpenAIInstrumentor())Use with OpenAI
from judgeval import Tracer
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from openai import OpenAI
Tracer.init(project_name="openllmetry_project")
Tracer.registerOTELInstrumentation(OpenAIInstrumentor())
client = OpenAI()
response = client.chat.completions.create(
model="gpt-5.2",
messages=[{"role": "user", "content": "Hello, world!"}]
)
print(response.choices[0].message.content)All OpenAI calls are automatically traced and exported to the Judgment platform.
Available Instrumentors
OpenLLMetry provides instrumentors for various frameworks and libraries:
OpenAI
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
Tracer.registerOTELInstrumentation(OpenAIInstrumentor())Anthropic
from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
Tracer.registerOTELInstrumentation(AnthropicInstrumentor())OpenAI Agents SDK
from opentelemetry.instrumentation.openai_agents import OpenAIAgentsInstrumentor
Tracer.registerOTELInstrumentation(OpenAIAgentsInstrumentor())Google Generative AI
from opentelemetry.instrumentation.google_generativeai import GoogleGenerativeAiInstrumentor
Tracer.registerOTELInstrumentation(GoogleGenerativeAiInstrumentor())Together AI
from opentelemetry.instrumentation.together import TogetherInstrumentor
Tracer.registerOTELInstrumentation(TogetherInstrumentor())Groq
from opentelemetry.instrumentation.groq import GroqInstrumentor
Tracer.registerOTELInstrumentation(GroqInstrumentor())HuggingFace Transformers
from opentelemetry.instrumentation.transformers import TransformersInstrumentor
Tracer.registerOTELInstrumentation(TransformersInstrumentor())Vertex AI
from opentelemetry.instrumentation.vertexai import VertexAIInstrumentor
Tracer.registerOTELInstrumentation(VertexAIInstrumentor())Ollama
from opentelemetry.instrumentation.ollama import OllamaInstrumentor
Tracer.registerOTELInstrumentation(OllamaInstrumentor())Other
Next Steps
- All Integrations - View the list of all integrations that Judgment supports.
- Agent Behavior Monitoring - Monitor your AI applications in production with behavioral scoring.
- Tracing Reference - Learn more about Judgment's tracing capabilities and advanced configuration.