Judgment Labs Logo

Pipelines

TypeScript JQL pipeline stages, execution order, and validation limits.

judgeval-js JQL source on GitHub

Call .pipe() before any select terminal. Stages execute in method-call order and each stage sees only the columns produced by the previous stage.

Pipelines require the traces or spans grain. Calling .pipe() on sessions fails with PIPE_GRAIN.

.where(filter: StageFilterInput): PipelineBuilder
.pick(options?: Omit<PickStage, "op">): PipelineBuilder
.derive(cols: Record<string, ExprInput>): PipelineBuilder
.summarize(aggs: Record<string, ExprInput>): PipelineBuilder
.summarize(by, aggs): PipelineBuilder
.sort(by: string): PipelineBuilder
.take(n: number, offset?: number): PipelineBuilder
.toJSON(): SourceQuery

Stage contract

StageInputs/defaultsOutput and constraints
wherefilterKeeps current rows. After summarize, acts like HAVING.
pickper?, by: "timestamp", n: 1, reverse: falseKeeps first/last rows per group; adds _rn. 1 <= n <= 1000.
derivecols: name -> expressionAdds columns without changing row count. Requires 1–16 outputs.
summarizeaggs: name -> expression, by?Collapses to one row per group. Requires 1–16 outputs and at most 4 grouping keys.
sortby: stringOrders current rows; append desc for descending.
taken, offset: 0Requires n >= 1, offset >= 0.

A pipeline has 1–8 stages in the validated JSON contract.

Stage-filter grammar

Pipeline where and conditional aggregate where do not accept the full root filter grammar. They accept only:

  • eq, ne, gt, gte, lt, lte
  • status
  • all, any, and not composed only from stage filters

Rich filters such as judge, grep, relation quantifiers, and nested source queries belong in the source/root filter before .pipe().

Output column names

derive and summarize output names must match:

^[a-z_][a-z0-9_]{0,63}$

_rn and bucket are reserved. _rn is produced by pick; bucket is produced by a bucket grouping expression.

Example and exact IR

import { aggExpr, spans } from "judgeval/jql";

const query = spans()
  .last("7d")
  .pipe()
  .summarize("model", { total_cost: aggExpr({ func: "sum", field: "cost" }) })
  .sort("total_cost desc")
  .take(10);
{
  "op": "query",
  "source": "spans",
  "time": { "last": "7d" },
  "pipe": [
    {
      "op": "summarize",
      "by": "model",
      "aggs": {
        "total_cost": { "op": "agg_expr", "func": "sum", "field": "cost" }
      }
    },
    { "op": "sort", "by": "total_cost desc" },
    { "op": "take", "n": 10 }
  ]
}

Referencing a removed or never-produced column fails with UNKNOWN_COLUMN and a hint listing the columns available at that stage.