# Judgment Labs

> Documentation for Judgment Labs: trace production AI agents, monitor behavior, build Agent Judges, run evaluations, and use the CLI, SDK, and MCP server.

## Get Started with Judgment

- [Get Started with Judgment](https://docs.judgmentlabs.ai/documentation)
- [Quickstart](https://docs.judgmentlabs.ai/documentation/quickstart): Instrument one real agent path, send a trace to Judgment, apply an Agent Judge, and inspect the result.
- [Judgment concepts](https://docs.judgmentlabs.ai/documentation/concepts): Learn how Judgment turns agent traces into production monitoring and offline tests.
- [Judgment Agent](https://docs.judgmentlabs.ai/documentation/judgment-agent): Understand how the in-product Judgment Agent combines page context, project data, and controlled actions.

### MCP and agent tools

- [Judgment MCP server](https://docs.judgmentlabs.ai/documentation/mcp-and-agent-tools): Give coding agents authenticated access to Judgment project data and tools.
- [Use Judgment from a coding agent](https://docs.judgmentlabs.ai/documentation/mcp-and-agent-tools/coding-agents): Combine Judgment project data and workflow guidance in a coding agent without confusing their roles.
- [Install the Judgment agent skill](https://docs.judgmentlabs.ai/documentation/mcp-and-agent-tools/judgment-agent-skill): Install Judgment tracing, evaluation, and SDK guidance into a coding agent for one repository and verify that the agent loads it.
- [MCP tools](https://docs.judgmentlabs.ai/documentation/mcp-and-agent-tools/mcp-tools): Reference for every currently registered Judgment data MCP tool, category, access level, and input field.
- [JQL](https://docs.judgmentlabs.ai/documentation/mcp-and-agent-tools/jql): Reference for querying, aggregating, and presenting Judgment trace, span, and session data.

### Tracing

- [Tracing data model](https://docs.judgmentlabs.ai/documentation/tracing): Understand how spans, traces, and sessions represent agent work in Judgment.
- [Instrument your agent](https://docs.judgmentlabs.ai/documentation/tracing/instrumentation): Initialize Judgment tracing, capture one real agent path, export it safely, and verify the trace.
- [Add attributes and context](https://docs.judgmentlabs.ai/documentation/tracing/attributes): Add queryable metadata, session and customer context, LLM usage, and handled errors to traced work.
- [Attribute keys](https://docs.judgmentlabs.ai/documentation/tracing/attribute-keys): Reference for the public judgment.* span attribute keys Judgment accepts as attribute-mapper targets.
- [Configure attribute mappers](https://docs.judgmentlabs.ai/documentation/tracing/attribute-mappers): Map incoming OpenTelemetry attributes onto Judgment's structured trace fields at ingest.
- [Trace streamed responses](https://docs.judgmentlabs.ai/documentation/tracing/streaming): Trace handlers that return a response stream before the model finishes.
- [Trace durable work](https://docs.judgmentlabs.ai/documentation/tracing/durable-work): Trace long-running loops, queued workers, workflow engines, and approval waits.
- [Trace subprocess models](https://docs.judgmentlabs.ai/documentation/tracing/subprocess-models): Trace models called through a CLI or subprocess.
- [Trace serverless runtimes](https://docs.judgmentlabs.ai/documentation/tracing/serverless-runtimes): Keep tracing alive through bundlers and runtimes that freeze.
- [Trace distributed agents](https://docs.judgmentlabs.ai/documentation/tracing/distributed): Keep one trace connected as a request crosses services.
- [Trace subagents](https://docs.judgmentlabs.ai/documentation/tracing/subagents): Keep delegated work in its parent trace or fork it into a linked trace when it needs independent evaluation.

### Integrations

- [Integrations](https://docs.judgmentlabs.ai/documentation/integrations): Choose the Judgment integration that matches how your application creates and exports traces.

#### Agent Frameworks

- [LangGraph Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/langgraph): Automatically trace Langgraph graph executions and node interactions.
- [OpenAI Agents SDK Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/openai-agents): Automatically trace OpenAI Agents SDK executions, handoffs, and agent interactions.
- [Claude Agent SDK Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/claude-agent-sdk): Automatically trace Claude Agent SDK executions, tool usage, and agent interactions.
- [Google ADK Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/google-adk): Automatically trace Google Agent Development Kit (ADK) executions, tool calls, and agent interactions.
- [Vercel AI SDK Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/vercel-ai-sdk): Automatically trace Vercel AI SDK agent executions, tool calls, and multi-step workflows.
- [LiveKit Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/livekit): Automatically trace LiveKit agent executions and interactions.
- [Pipecat Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/pipecat): Automatically trace Pipecat voice and multimodal agent pipelines.
- [OpenClaw Tracing](https://docs.judgmentlabs.ai/documentation/integrations/agent-frameworks/openclaw): Send OpenClaw Gateway traces to the Judgment platform via OpenTelemetry.

#### Model Providers

- [Anthropic Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/anthropic)
- [OpenAI Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/openai)
- [OpenRouter Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/openrouter)
- [Google Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/google)
- [Groq Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/groq)
- [Together AI Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/together_ai)
- [HuggingFace Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/huggingface)
- [Vertex AI Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/vertexai)
- [Ollama Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/ollama)
- [AWS Bedrock Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/aws_bedrock)
- [LiteLLM Integration](https://docs.judgmentlabs.ai/documentation/integrations/model-providers/litellm)

#### Tracing Providers

- [Direct OTEL Trace Export](https://docs.judgmentlabs.ai/documentation/integrations/tracing-providers/direct-otel): Export standard OTLP/HTTP traces directly to Judgment.
- [Dual Export Trace Export](https://docs.judgmentlabs.ai/documentation/integrations/tracing-providers/dual-export): Send traces from existing observability providers to Judgment.
- [OpenLit Trace Export](https://docs.judgmentlabs.ai/documentation/integrations/tracing-providers/openlit): Export OpenLit traces to the Judgment platform.
- [OpenInference Trace Export](https://docs.judgmentlabs.ai/documentation/integrations/tracing-providers/openinference): Export OpenInference traces to the Judgment platform.
- [OpenLLMetry Trace Export](https://docs.judgmentlabs.ai/documentation/integrations/tracing-providers/openllmetry): Export OpenLLMetry traces to the Judgment platform.

#### Other

- [FastAPI Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/fastapi): Trace FastAPI request spans with Judgment.
- [Starlette Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/starlette): Trace Starlette request spans with Judgment.
- [Flask Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/flask): Trace Flask request spans with Judgment.
- [Django Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/django): Trace Django request spans with Judgment.
- [AIOHTTP Server Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/aiohttp-server): Trace AIOHTTP server request spans with Judgment.
- [ASGI Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/asgi): Trace generic ASGI applications with Judgment.
- [WSGI Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/wsgi): Trace generic WSGI applications with Judgment.
- [HTTPX Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/httpx): Trace HTTPX requests with Judgment.
- [Requests Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/requests): Trace Requests HTTP calls with Judgment.
- [AIOHTTP Client Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/aiohttp-client): Trace AIOHTTP client requests with Judgment.
- [SQLAlchemy Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/sqlalchemy): Trace SQLAlchemy database spans with Judgment.
- [Psycopg Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/psycopg): Trace Psycopg PostgreSQL spans with Judgment.
- [Asyncpg Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/asyncpg): Trace Asyncpg PostgreSQL spans with Judgment.
- [PyMongo Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/pymongo): Trace PyMongo spans with Judgment.
- [MySQL Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/mysql): Trace MySQL spans with Judgment.
- [SQLite3 Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/sqlite3): Trace SQLite3 spans with Judgment.
- [Redis Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/redis): Trace Redis spans with Judgment.
- [Celery Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/celery): Trace Celery worker spans with Judgment.
- [Temporal Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/temporal): Trace Temporal workflow and activity spans with Judgment.
- [AWS Lambda Tracing](https://docs.judgmentlabs.ai/documentation/integrations/other/aws-lambda): Trace AWS Lambda invocations with Judgment.

### Monitoring

- [Production monitoring](https://docs.judgmentlabs.ai/documentation/monitoring): Track known agent behaviors across live traffic and inspect the traces behind them.
- [Investigate a failed trace](https://docs.judgmentlabs.ai/documentation/monitoring/trace-investigation): Inspect a production trace from its failure signal through span evidence and save the case for regression testing.
- [Investigate production issues](https://docs.judgmentlabs.ai/documentation/monitoring/production-issues): Understand how Judgment discovers and tracks unknown recurring problems across production traces.
- [Configure automations and alerts](https://docs.judgmentlabs.ai/documentation/monitoring/automations-and-alerts): Run actions when a completed trace matches behavior or trace conditions.

### Judges

- [Evaluation data model](https://docs.judgmentlabs.ai/documentation/judges): Understand how judges, outputs, results, and behaviors represent evaluation in Judgment.
- [Agent Judges](https://docs.judgmentlabs.ai/documentation/judges/agent-judges): Use natural-language rubrics to evaluate traces and examples with structured results and supporting evidence.
- [Code Judges](https://docs.judgmentlabs.ai/documentation/judges/code-judges): Use Python to evaluate examples with deterministic logic, libraries, or custom model calls.
- [Behaviors](https://docs.judgmentlabs.ai/documentation/judges/behavior-setup): Track binary or categorical judge outputs across production traffic.

### Datasets

- [Datasets](https://docs.judgmentlabs.ai/documentation/datasets): Understand how datasets preserve reusable examples and production evidence for judge calibration and offline tests.
- [Create a dataset](https://docs.judgmentlabs.ai/documentation/datasets/creation): Create a dataset in the Judgment platform and define the schema every example must follow.
- [Add production traces to a dataset](https://docs.judgmentlabs.ai/documentation/datasets/production-traces): Map production trace evidence into schema-enforced dataset rows for review and regression testing.
- [Manage dataset versions](https://docs.judgmentlabs.ai/documentation/datasets/versions): Create stable dataset membership snapshots, inspect earlier versions, and choose the version an offline test runs.

### Tests

- [Run an offline test](https://docs.judgmentlabs.ai/documentation/tests/offline-tests): Evaluate one dataset version with selected judges and inspect the pinned results in Judgment.
- [Calibrate a judge](https://docs.judgmentlabs.ai/documentation/tests/judge-calibration): Review a binary or categorical judge against human judgment and use disagreements to refine it.
- [Compare test runs](https://docs.judgmentlabs.ai/documentation/tests/test-run-comparison): Compare two runs from the same test config and localize result changes to examples, traces, and judge versions.
- [Turn production failures into regression tests](https://docs.judgmentlabs.ai/documentation/tests/regression-cases): Carry one failed production trace into a dataset, a repeatable test, and a baseline-versus-candidate comparison.

### Reference

- [Authentication](https://docs.judgmentlabs.ai/documentation/reference/authentication): Reference for Judgment API keys, organization identity, and the clients that support OAuth.
- [Project routing](https://docs.judgmentlabs.ai/documentation/reference/project-routing): Reference for selecting Judgment projects by name or ID and separating deployment environments safely.
- [Trace conventions](https://docs.judgmentlabs.ai/documentation/reference/trace-conventions): Reference for OpenTelemetry resource, context, root-span, and Judgment attribute conventions.
- [Rate Limits](https://docs.judgmentlabs.ai/documentation/reference/rate-limits): Default request rate limits for Judgment's platform APIs.

## SDK Reference

- [Judgeval SDK Reference](https://docs.judgmentlabs.ai/sdk-reference): Reference documentation for the Judgeval SDK

### Python

- [Judgeval](https://docs.judgmentlabs.ai/sdk-reference/python/judgeval): The main entry point for interacting with the Judgment platform.
- [Judge](https://docs.judgmentlabs.ai/sdk-reference/python/judges): Base class for building custom evaluation scorers.

#### JQL

- [JQL](https://docs.judgmentlabs.ai/sdk-reference/python/jql): Python SDK reference for the complete Judgment Query Language contract.
- [Client methods](https://docs.judgmentlabs.ai/sdk-reference/python/jql/client): Python JQL client methods, request options, and transport behavior.
- [Queries and terminals](https://docs.judgmentlabs.ai/sdk-reference/python/jql/queries): Python JQL roots, time bounds, select terminals, and result row shapes.
- [Filters and expressions](https://docs.judgmentlabs.ai/sdk-reference/python/jql/filters-and-expressions): Python JQL filter and expression constructors with exact validation rules.
- [Pipelines](https://docs.judgmentlabs.ai/sdk-reference/python/jql/pipelines): Python JQL pipeline stages, execution order, and validation limits.
- [Presentations](https://docs.judgmentlabs.ai/sdk-reference/python/jql/presentations): Python JQL chart and table inputs and typed presentation-frame output.
- [Discovery](https://docs.judgmentlabs.ai/sdk-reference/python/jql/discovery): Python JQL discovery kinds, options, and returned catalog rows.
- [Responses and errors](https://docs.judgmentlabs.ai/sdk-reference/python/jql/responses-and-errors): Python JQL public response envelopes, nullability, and error handling.
- [PipelineBuilder](https://docs.judgmentlabs.ai/sdk-reference/python/jql/PipelineBuilder)
- [QueryBuilder](https://docs.judgmentlabs.ai/sdk-reference/python/jql/QueryBuilder)

#### Trace

- [OfflineTracer](https://docs.judgmentlabs.ai/sdk-reference/python/trace/offline_tracer): Tracer for offline / experiment-style runs.
- [Tracer](https://docs.judgmentlabs.ai/sdk-reference/python/trace/tracer): Capture execution traces and LLM performance metrics for your application.
- [LinkedTraceSpans](https://docs.judgmentlabs.ai/sdk-reference/python/trace/internal)
- [Propagation](https://docs.judgmentlabs.ai/sdk-reference/python/trace/propagation)

##### Judgment Tracer Provider

- [ProxyTracer](https://docs.judgmentlabs.ai/sdk-reference/python/trace/judgment_tracer_provider/ProxyTracer): Internal tracer that delegates to the currently active ``JudgmentTracer``.
- [JudgmentTracerProvider](https://docs.judgmentlabs.ai/sdk-reference/python/trace/judgment_tracer_provider/JudgmentTracerProvider): Global singleton that manages Judgment tracers and context propagation.

##### Exporters

- [JudgmentSpanExporter](https://docs.judgmentlabs.ai/sdk-reference/python/trace/exporters/judgment_span_exporter): Exports completed spans to the Judgment platform over OTLP/HTTP.
- [NoOpJudgmentSpanExporter](https://docs.judgmentlabs.ai/sdk-reference/python/trace/exporters/noop_judgment_span_exporter): A span exporter that silently discards all spans.

##### Processors

- [OfflineJudgmentSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/python/trace/processors/offline_judgment_span_processor): Span processor used by ``OfflineTracer``.
- [JudgmentBaggageProcessor](https://docs.judgmentlabs.ai/sdk-reference/python/trace/processors/judgment_baggage_processor): Copies OTel baggage entries onto span attributes at span start.
- [NoOpJudgmentSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/python/trace/processors/noop_judgment_span_processor): A span processor that silently discards all operations.
- [JudgmentSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/python/trace/processors/judgment_span_processor): Span processor that manages span lifecycle, state, and batched export.

##### Baggage

- [Baggage](https://docs.judgmentlabs.ai/sdk-reference/python/trace/baggage): Reference for get_all, get_baggage, set_baggage, remove_baggage, clear, and Propagator.
- [JudgmentBaggagePropagator](https://docs.judgmentlabs.ai/sdk-reference/python/trace/baggage/propagator): W3C Baggage propagator that uses the Judgment context.

#### Hosted

- [TraceCustomScorer](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/trace_custom_scorer): **Deprecated.** Use `Judge[R]` from `judgeval.judges` instead.
- [ExampleCustomScorer](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/example_custom_scorer): **Deprecated.** Use `Judge[R]` from `judgeval.judges` instead.

##### Responses

- [Responses](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses): Reference for Citation, Category, BaseResponse, BinaryResponse, CategoricalResponse, and NumericResponse.
- [Citation](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/Citation): Links a score back to a specific span in a trace.
- [Category](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/Category): Defines one allowed category for a `CategoricalResponse` scorer.
- [BaseResponse](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/BaseResponse): Base class for all scorer response types.
- [BinaryResponse](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/BinaryResponse): Pass/fail response for binary scorers.
- [CategoricalResponse](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/CategoricalResponse): Response for classification-style scorers.
- [NumericResponse](https://docs.judgmentlabs.ai/sdk-reference/python/hosted/responses/NumericResponse): Response for numeric scorers (e.g. 0.0 to 1.0).

#### Agent Judges

- [AgentJudge](https://docs.judgmentlabs.ai/sdk-reference/python/agent_judges/agent_judge): A prompt-based Agent Judge stored on the Judgment platform.
- [AgentJudgeFactory](https://docs.judgmentlabs.ai/sdk-reference/python/agent_judges/agent_judge_factory): Create and update prompt-based Agent Judges on the Judgment platform.

#### Datasets

##### Dataset Factory

- [Dataset Factory](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset_factory): Reference for DatasetSchemaProperty, DatasetSchema, DatasetFactory, validate_dataset_schema, and infer_schema_from_examples.
- [DatasetSchemaProperty](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset_factory/DatasetSchemaProperty): A single dataset column declaration.
- [DatasetSchema](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset_factory/DatasetSchema): A dataset's JSON Schema.
- [DatasetFactory](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset_factory/DatasetFactory): Create, retrieve, list, and delete datasets in your project.

##### Dataset

- [Dataset](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset): Reference for DatasetInfo, DatasetVersion, Dataset, example_to_dataset_entry, and example_from_dataset_entry.
- [DatasetInfo](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset/DatasetInfo): Summary metadata returned when listing datasets.
- [DatasetVersion](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset/DatasetVersion): A single immutable version of a dataset.
- [Dataset](https://docs.judgmentlabs.ai/sdk-reference/python/datasets/dataset/Dataset): A schema-enforced collection of `Example` objects on the Judgment platform.

#### Integrations

- [Langgraph](https://docs.judgmentlabs.ai/sdk-reference/python/integrations/langgraph): Integration that routes LangGraph spans through the Judgment pipeline.
- [Claude Agent SDK](https://docs.judgmentlabs.ai/sdk-reference/python/integrations/claude_agent_sdk)
- [Openlit](https://docs.judgmentlabs.ai/sdk-reference/python/integrations/openlit): Integration that routes OpenLit instrumentation through Judgment.

#### Prompts

- [Prompt](https://docs.judgmentlabs.ai/sdk-reference/python/prompts/prompt): A versioned prompt template with variable substitution.
- [PromptFactory](https://docs.judgmentlabs.ai/sdk-reference/python/prompts/prompt_factory): Create, retrieve, tag, and list versioned prompt templates.

#### Evaluation

- [HostedEvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/python/evaluation/hosted_evaluation)
- [LocalEvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/python/evaluation/local_evaluation)
- [Evaluation](https://docs.judgmentlabs.ai/sdk-reference/python/evaluation/evaluation): Score a batch of examples using hosted scorers or custom judges.
- [EvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/python/evaluation/evaluation_base): Abstract base for evaluation runners.
- [EvaluationFactory](https://docs.judgmentlabs.ai/sdk-reference/python/evaluation/evaluation_factory): Creates `Evaluation` instances for running batch scoring.

#### Offline Tests

- [OfflineTestsFactory](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/offline_tests_factory): Create test configs and execute offline test runs.

##### Types

- [TestConfig](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/types/TestConfig): A reusable offline-test configuration (dataset + judges).
- [OfflineTestResult](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/types/OfflineTestResult): The outcome of an offline test run.

##### Offline Test Runner

- [Offline Test Runner](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/offline_test_runner): Reference for JudgeVersionPin, OfflineTestRunner, normalize_judge_versions, and build_agent_kwargs.
- [JudgeVersionPin](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/offline_test_runner/JudgeVersionPin): A single ``judge_versions`` entry.
- [OfflineTestRunner](https://docs.judgmentlabs.ai/sdk-reference/python/offline_tests/offline_test_runner/OfflineTestRunner): Executes the offline-test lifecycle for a test config.

#### Data

- [Trace](https://docs.judgmentlabs.ai/sdk-reference/python/data/trace): A recorded execution trace consisting of one or more spans.
- [ScorerData](https://docs.judgmentlabs.ai/sdk-reference/python/data/scorer_data): The result from a single scorer for one example.
- [Example](https://docs.judgmentlabs.ai/sdk-reference/python/data/example): A single evaluation example with flexible key-value properties.
- [ScoringResult](https://docs.judgmentlabs.ai/sdk-reference/python/data/scoring_result): The combined result of running scorers against a single example.

### TypeScript

- [Judgeval](https://docs.judgmentlabs.ai/sdk-reference/typescript/judgeval): The main entry point for interacting with the Judgment platform.
- [JudgevalConfig](https://docs.judgmentlabs.ai/sdk-reference/typescript/judgeval-config): Configuration options for the Judgeval client.
- [JudgevalAPIError](https://docs.judgmentlabs.ai/sdk-reference/typescript/judgeval-api-error)
- [JqlRequestOptions](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql-request-options)

#### JQL

- [JQL](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql): TypeScript SDK reference for the complete Judgment Query Language contract.
- [Client methods](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/client): TypeScript JQL client methods, request options, and transport behavior.
- [Queries and terminals](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/queries): TypeScript JQL roots, time bounds, select terminals, and result row shapes.
- [Filters and expressions](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/filters-and-expressions): TypeScript JQL filter and expression constructors with exact validation rules.
- [Pipelines](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/pipelines): TypeScript JQL pipeline stages, execution order, and validation limits.
- [Presentations](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/presentations): TypeScript JQL chart and table inputs and typed presentation-frame output.
- [Discovery](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/discovery): TypeScript JQL discovery kinds, options, and returned catalog rows.
- [Responses and errors](https://docs.judgmentlabs.ai/sdk-reference/typescript/jql/responses-and-errors): TypeScript JQL public response envelopes, nullability, and error handling.

#### Datasets

- [DatasetInfo](https://docs.judgmentlabs.ai/sdk-reference/typescript/datasets/dataset-info)
- [DatasetSchemaProperty](https://docs.judgmentlabs.ai/sdk-reference/typescript/datasets/dataset-schema-property): A single dataset column declaration.
- [DatasetSchema](https://docs.judgmentlabs.ai/sdk-reference/typescript/datasets/dataset-schema): A dataset's JSON Schema. Datasets are object-typed with one property per column. A column declared `{ type: "trace" }` (under any name) holds a trace id; at most one trace column is allowed per dataset.
- [Dataset](https://docs.judgmentlabs.ai/sdk-reference/typescript/datasets/dataset): A collection of Example objects stored on the Judgment platform.
- [DatasetFactory](https://docs.judgmentlabs.ai/sdk-reference/typescript/datasets/dataset-factory): Creates, retrieves, and lists datasets in your project.

#### Evaluation

- [ExperimentScorer](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/experiment-scorer)
- [ScoringResult](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/scoring-result): The combined result of running scorers against a single example.
- [Evaluation](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/evaluation): Score a batch of examples using hosted scorers or custom judges.
- [EvaluationRunOptions](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/evaluation-run-options)
- [AsyncEvaluateOptions](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/async-evaluate-options): Options for BaseTracer.asyncEvaluate.
- [EvaluationFactory](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/evaluation-factory): Creates Evaluation instances for running batch scoring.
- [EvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/evaluator-runner): Abstract base for evaluation runners.
- [LocalEvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/local-evaluator-runner): Evaluation runner for custom (in-process) scorers.
- [HostedEvaluatorRunner](https://docs.judgmentlabs.ai/sdk-reference/typescript/evaluation/hosted-evaluator-runner): Evaluation runner for hosted (server-side) scorers.

#### Trace

- [JudgmentBaggageSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/judgment-baggage-span-processor): Span processor that copies baggage entries onto span attributes at span start. Use `keyPredicate` to control which keys are propagated.
- [JudgmentSpanExporter](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/judgment-span-exporter): Span exporter that sends traces to the Judgment platform via OTLP HTTP.
- [JudgmentTracerProvider](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/judgment-tracer-provider): Global singleton that manages tracer registration and context propagation.
- [Tracer](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/tracer): Concrete tracer implementation for Node.js applications.
- [NoOpSpanExporter](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/no-op-span-exporter): A no-op span exporter that discards all spans.
- [NoOpSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/no-op-span-processor): A no-op span processor that discards all spans.
- [LLMMetadata](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/llm-metadata): Metadata about an LLM call to record on a span.
- [ObserveOptions](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/observe-options): Options for BaseTracer.observe.
- [TracerConfig](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/tracer-config): Configuration options for initializing a Tracer.
- [JudgmentSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/typescript/trace/judgment-span-processor): Span processor that manages span lifecycle, state, and batched export to the Judgment platform. Supports per-span state (counters, lists), partial-span emission for streaming updates, and baggage propagation onto child spans.

#### Offline Tests

- [OfflineTracer](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-tracer): Tracer for offline / experiment-style runs.
- [OfflineTracerConfig](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-tracer-config): Configuration for `OfflineTracer.create()`.
- [OfflineJudgmentSpanProcessor](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-judgment-span-processor): Span processor used by `OfflineTracer`.
- [OfflineTestsFactory](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-tests-factory): Create test configs and execute offline test runs. Access via `client.offlineTests`. A *test config* pairs a dataset with a set of platform judges; a *test run* evaluates one dataset version and stores per-example results.
- [OfflineTestRunner](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-test-runner): Executes the offline-test lifecycle for a test config: resolve the dataset version, optionally run the agent to produce offline traces, create the test run, wait for terminal status, fetch results, evaluate the pass condition, and report successes.
- [OfflineRunOptions](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-run-options): Options accepted by OfflineTestRunner.run.
- [TestConfig](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/test-config): Types for the offline-tests SDK surface: TestConfig, OfflineTestResult, JudgeVersionPin, AgentFunction, PassConditionFn.
- [OfflineScorerData](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-scorer-data): One judge's result for one example.
- [OfflineExampleResult](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-example-result): Per-example results for an offline test run.
- [OfflineTestResult](https://docs.judgmentlabs.ai/sdk-reference/typescript/offline_tests/offline-test-result): The outcome of an offline test run, returned by `client.offlineTests.run()`.

#### Data

- [Example](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/example): A single evaluation example with flexible key-value properties.
- [BinaryResponse](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/binary-response): Pass/fail response for binary scorers.
- [NumericResponse](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/numeric-response): Numeric score response (e.g. 0.0 to 1.0).
- [CategoricalResponse](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/categorical-response): Classification-style response for categorical scorers.
- [Citation](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/citation): Links a score back to a specific span in a trace, enabling drill-down from evaluation results into the traced execution.
- [BaseResponse](https://docs.judgmentlabs.ai/sdk-reference/typescript/data/base-response): Base fields shared by all scorer response types.

#### Judges

- [Judge](https://docs.judgmentlabs.ai/sdk-reference/typescript/judges/judge): Base class for building custom evaluation scorers.
- [AgentJudgeFactory](https://docs.judgmentlabs.ai/sdk-reference/typescript/judges/agent-judge-factory): Create and update prompt-based Agent Judges on the Judgment platform.
- [AgentJudge](https://docs.judgmentlabs.ai/sdk-reference/typescript/judges/agent-judge)
- [JudgeVersionPin](https://docs.judgmentlabs.ai/sdk-reference/typescript/judges/judge-version-pin): A single `judgeVersions` entry. Identify the judge by `name` or `judgeId`; optionally pin a `tag`, a `version` string, or a `majorVersion`/`minorVersion` pair. Every entry must carry a `name` or `judgeId`.

## CLI Reference

- [CLI Reference](https://docs.judgmentlabs.ai/cli-reference): Legacy command reference for the deprecated Judgment CLI.
- [judgment login](https://docs.judgmentlabs.ai/cli-reference/login): Authenticate and store credentials locally.
- [judgment logout](https://docs.judgmentlabs.ai/cli-reference/logout): Remove stored credentials.
- [judgment status](https://docs.judgmentlabs.ai/cli-reference/status): Show current authentication status and credential sources.
- [judgment configure](https://docs.judgmentlabs.ai/cli-reference/configure): Update stored credentials interactively.
- [judgment completion](https://docs.judgmentlabs.ai/cli-reference/completion): Print a shell-completion script for SHELL.
- [judgment agent-threads](https://docs.judgmentlabs.ai/cli-reference/agent-threads): List and inspect agent thread conversations (global_copilot, custom_agent).
- [judgment automations](https://docs.judgmentlabs.ai/cli-reference/automations): Manage automations (rules) that fire actions when metrics match conditions.
- [judgment context](https://docs.judgmentlabs.ai/cli-reference/context): Manage the default organization and project for commands.
- [judgment docs](https://docs.judgmentlabs.ai/cli-reference/docs): Search and read Judgment documentation.
- [judgment organizations](https://docs.judgmentlabs.ai/cli-reference/organizations): List the organizations the authenticated user can access.
- [judgment projects](https://docs.judgmentlabs.ai/cli-reference/projects): Manage projects in your organization.
- [judgment prompts](https://docs.judgmentlabs.ai/cli-reference/prompts): Fetch and commit versioned prompts in a project, including tagging commits (e.g. `production`).
- [judgment sessions](https://docs.judgmentlabs.ai/cli-reference/sessions): Search and inspect sessions in a project.
- [judgment traces](https://docs.judgmentlabs.ai/cli-reference/traces): Search, inspect, tag, and re-evaluate traces in a project.