Judgment Labs Logo
PythonJQL

Filters and expressions

Python JQL filter and expression constructors with exact validation rules.

judgeval JQL source on GitHub

Public filter constructors

status(value)
name(value)
model(value)
cost(**bounds)
duration(**bounds)
judge(name, value=None)
judged(*, name=None, value=None, prompt=None, type=None, mode=None)
attr(key, value=None, selector=None)
grep(field, value)
rg(field, pattern, *, ignore_case=None)
tokens(field, words)
eq(field, value) / ne(...) / gt(...) / gte(...) / lt(...) / lte(...)
cited_by(judge, value=None)
all_(first, *rest) / any_(first, *rest) / not_(filter)
any_span(filter) / every_span(filter) / no_span(filter)
any_trace(filter) / every_trace(filter) / no_trace(filter)
descendant_of(filter, depth=1) / ancestor_of(filter, depth=1)
over_spans(agg, cmp, value, where=None)
over_traces(agg, cmp, value, where=None)
over_scores(agg, cmp, value)
at_least(k, of, where=None)

Leaf filters

OperationRequired inputsOptional inputsConstraints
statusvalue: "error" | "ok"Closed enum; any other value is rejected with BAD_STATUS. Valid on all grains
namevalue: stringMatches span_name; valid on all grains
modelvalue: stringSpan grain
costAt least one boundgt, gte, lt, lteNumeric; span grain
durationAt least one boundgt, gte, lt, lteNumeric nanoseconds
judgename: stringvalue: anyRaw score facts
judgedAt least one of name, prompt, type, modevaluevalue alone is rejected. prompt must match [\w \-]+
attrkey: stringvalue, selectorStructured attribute lookup
grepfield, valueSubstring search
rgfield, patternignore_case (wire) / ignoreCaseRegex search
tokensfield, wordsWhole-word search; words are whitespace-split and each must match [A-Za-z0-9]+
comparisonsfield, valueeq ne gt gte lt lte; value may be a literal or col(...)
cited_byjudgevalueSpans cited as judge evidence

Text operations are supported on input, output, span_name, and error, subject to the grain: error is not a span field, so text operations on it fail with FIELD_NOT_ON_GRAIN at the spans grain. grep, rg, and tokens reject an unsupported field at construction time, not only at the server.

Boolean, relation, and hierarchy filters

  • all_ and any_ require at least one filter. not_ takes exactly one.
  • any_span, every_span, and no_span quantify child spans.
  • any_trace, every_trace, and no_trace quantify traces in a session.
  • A complete source query may be nested as a cross-grain filter.
  • descendant_of and ancestor_of accept depth=1 (the default) or null only, and are valid only on the spans grain.
  • at_least requires integer k >= 1; of is spans or traces.

Aggregate relation predicates

over_spans, over_traces, and over_scores compare an aggregate with a numeric threshold through eq|ne|gt|gte|lt|lte.

The aggregate object is:

{ "func": "count" }

or:

{ "func": "sum", "field": "cost" }

func is count, count_distinct, sum, avg, min, or max. field is required except for count. over_scores does not accept where; filter the outer grain instead. over_scores also aggregates over the scores relation, so it accepts only that relation's own fields, not arbitrary grain fields — an unsupported field fails with BAD_FIELD.

Expressions

Expressions are values used by derive and summarize, not root filters.

col(name) -> dict
agg_expr(func, field=None, *, q=None, per=None, where=None) -> dict
arith(fn, left, right) -> dict
bucket(field, every) -> dict

The table below gives the canonical JSON each constructor emits. The op values are language-neutral and always snake_case, regardless of the constructor name.

ConstructorEmitted JSONRules
col{"op":"col","name":"cost"}References a current pipeline column
agg_expr{"op":"agg_expr","func":"sum","field":"cost","per":"trace","where":...}See aggregate-expression rules below
arith{"op":"arith","fn":"div","left":...,"right":...}fn: div mul add sub; operands are expressions or numbers
bucket{"op":"bucket","field":"timestamp","every":"1d"}Valid only as a summarize grouping key; produces bucket

Aggregate expressions support count, count_distinct, sum, avg, min, max, and quantile. All except count require field. quantile requires q with 0 < q < 1 and does not support where. q is rejected for non-quantile functions. where uses the narrow pipeline stage-filter grammar. per creates a per-group window aggregate.

Nested arith expressions are capped at 8 levels inside a single derive or summarize output; the limit counts arithmetic nesting only and does not descend into aggregate operands. Boolean values are not numeric expression literals.