judgeval
Judgeval
JudgevalConfig
DatasetInfo
ExperimentScorer
JudgmentBaggageSpanProcessor
JudgmentSpanExporter
JudgmentTracerProvider
Tracer
OfflineTracer
OfflineTracerConfig
NoOpSpanExporter
NoOpSpanProcessor
OfflineJudgmentSpanProcessor
Example
ScoringResult
Judge
BinaryResponse
NumericResponse
CategoricalResponse
Citation
Evaluation
EvaluationRunOptions
Dataset
LLMMetadata
ObserveOptions
AsyncEvaluateOptions
TracerConfig
JudgmentSpanProcessor
BaseResponse
EvaluationFactory
EvaluatorRunner
LocalEvaluatorRunner
HostedEvaluatorRunner
DatasetFactory
wrap()
Wrap a supported LLM client to add automatic tracing.
Currently supports OpenAI clients. Detects the client type automatically and applies the appropriate instrumentation.
import OpenAI from "openai";
import { wrap } from "judgeval";
const client = wrap(new OpenAI());function wrap(client: T): TParameters
client
required:T
An OpenAI client instance.
Returns
T - The same client instance, instrumented in-place.
wrapOpenAI()
Instrument an OpenAI client instance to emit Judgment spans.
Patches the following methods in-place:
client.chat.completions.create(streaming + non-streaming)client.chat.completions.parse(non-streaming)client.responses.create(streaming + non-streaming)client.images.generate(streaming + non-streaming)
function wrapOpenAI(client: T): TParameters
client
required:T
Returns
T - The same client instance (mutated).
getGlobalTextmap()
Return the active composite propagator (W3C TraceContext + Judgment Baggage).
function getGlobalTextmap(): TextMapPropagator<any>Returns
TextMapPropagator<any>
setGlobalTextmap()
Replace the global text-map propagator.
function setGlobalTextmap(propagator: TextMapPropagator<any>): voidParameters
propagator
required:TextMapPropagator<any>
Returns
void
inject()
Inject trace context and baggage into an outgoing carrier (e.g. HTTP headers).
Call this before making an outbound HTTP request to propagate the current trace across service boundaries.
const headers: Record<string, string> = {};
propagation.inject(headers);
await fetch("https://api.example.com", { headers });function inject(carrier: Carrier, context?: Context | undefined, setter: TextMapSetter<Carrier> = defaultTextMapSetter as TextMapSetter<Carrier>): voidParameters
carrier
required:Carrier
A mutable object to write propagation headers into.
context
:Context | undefined
The context to inject. Defaults to the current Judgment context.
setter
:TextMapSetter<Carrier>
Strategy for writing values into the carrier.
defaultTextMapSetter as TextMapSetter<Carrier>
Returns
void
extract()
Extract trace context and baggage from an incoming carrier.
Low-level primitive — most callers should use Tracer.continueTrace instead, which extracts and installs the context in one step.
function extract(carrier: Carrier, context?: Context | undefined, getter: TextMapGetter<Carrier> = defaultTextMapGetter as TextMapGetter<Carrier>): ContextParameters
carrier
required:Carrier
A mapping containing propagation headers (e.g.
request.headers).
context
:Context | undefined
Base context to merge into. Defaults to the current active context.
getter
:TextMapGetter<Carrier>
Strategy for reading values from the carrier.
defaultTextMapGetter as TextMapGetter<Carrier>
Returns
Context - A new Context with the extracted trace and baggage
data.
getBaggage()
Retrieve the baggage attached to the given context.
function getBaggage(context: Context): Baggage | undefinedParameters
context
required:Context
Returns
Baggage | undefined
getActiveBaggage()
Retrieve the baggage attached to the active context.
function getActiveBaggage(): Baggage | undefinedReturns
Baggage | undefined
setBaggage()
Attach a baggage to the given context, returning a new context.
function setBaggage(context: Context, baggage: Baggage): ContextParameters
context
required:Context
baggage
required:Baggage
Returns
Context
deleteBaggage()
Remove the baggage attached to the given context, returning a new context.
function deleteBaggage(context: Context): ContextParameters
context
required:Context
Returns
Context
baggageEntryMetadataFromString()
Create a serializable BaggageEntryMetadata object from a string.
function baggageEntryMetadataFromString(str: string): BaggageEntryMetadataParameters
str
required:string
string metadata. Format is currently not defined by the spec and has no special meaning.
Returns
BaggageEntryMetadata