---
title: "Observations"
sidebar:
  label: "Observations"
  badge: "Beta"
description: "Record each model call your agent makes as an observation: the JSON format and the span attribute that carries it."
seo:
  title: "Observations | Judgment Docs"
  description: "Record each model call your agent makes as an observation: the JSON format and the span attribute that carries it."
---

An observation records one model call your agent makes: everything the model was given and what it produced. Judgment groups observations into conversations and shows them, exactly as the model saw them, in the Conversation view. Supported integrations record observations automatically. For any other model call, send the observation yourself.

> **Info**
>
> Observations are in beta. Judgment records them only for organizations where they're turned on. [Contact support](/documentation/start/get-help) to turn them on.

## Send an observation

Set the observation on the model call's span. It travels as JSON text under `judgment.conversation.observation`. One span holds one observation; setting it again replaces it.

**Python**

`Tracer.set_conversation_observation` takes a typed `ConversationObservation`, so your type checker flags fields the format doesn't define before Judgment rejects them.

```python
from judgeval import Tracer
from judgeval.trace import ConversationMessage, ConversationObservation

@Tracer.observe(span_type="llm", span_name="support_reply")
def reply(history: list[ConversationMessage]) -> str:
    response = my_model.generate(history)
    observation: ConversationObservation = {
        "model": "my-model-v2",
        "input": [
            {"type": "message", "role": "system", "content": [{"type": "text", "text": "You help customers with orders."}]},
            *history,
        ],
        "output": [
            {"type": "message", "role": "assistant", "content": [{"type": "text", "text": response.text}]},
        ],
        "finish_reason": "stop",
    }
    Tracer.set_conversation_observation(observation)
    return response.text
```

**TypeScript**

`Tracer.setConversationObservation` takes a typed `ConversationObservation`, so the compiler flags fields the format doesn't define before Judgment rejects them.

```typescript
import { Tracer, type ConversationMessage, type ConversationObservation } from "judgeval";

const reply = Tracer.observe(async (history: ConversationMessage[]) => {
  const response = await myModel.generate(history);
  const observation: ConversationObservation = {
    model: "my-model-v2",
    input: [
      { type: "message", role: "system", content: [{ type: "text", text: "You help customers with orders." }] },
      ...history,
    ],
    output: [
      { type: "message", role: "assistant", content: [{ type: "text", text: response.text }] },
    ],
    finish_reason: "stop",
  };
  Tracer.setConversationObservation(observation);
  return response.text;
}, { spanType: "llm", spanName: "support_reply" });
```

**OpenTelemetry**

```python
import json
from opentelemetry import trace

span = trace.get_current_span()
span.set_attribute("judgment.conversation.observation", json.dumps(observation))
```

## Observation

| Field | Type | Meaning |
| --- | --- | --- |
| `input` | Item list, required | Everything the model was given, in order. |
| `output` | Item list | What the model produced, when recorded. |
| `conversation_id` | String | The thread of context the call belongs to. Defaults to the span's `judgment.session_id`. One of the two is required. |
| `customer_id` | String | The customer the call ran for. Defaults to the span's `judgment.customer_id`. |
| `user_id` | String | The user the call ran for. Defaults to the span's `judgment.customer_user_id`. |
| `tools` | Tool list | The tools the model could call: `name`, and optional `description`, `parameters` (JSON Schema of the arguments), and `metadata`. |
| `model` | String | The model that ran the call. |
| `finish_reason` | String | Why the model stopped, as the provider reported it. |
| `usage` | Usage | What the call consumed. Defaults to the span's `judgment.usage.*` attributes. |
| `error` | Error | A recorded model-call failure. |
| `metadata` | Object of JSON values | Any other facts about this call. |

Set `conversation_id` (or the session ID) to the same value on every call that continues the same context. See [Session guidance](/documentation/connect/add-context#sessions-customers-and-end-users).

## Items

`input` and `output` are flat, ordered lists of items. An assistant turn with text and two tool calls is one `message` item followed by two `tool_call` items.

| `type` | Fields |
| --- | --- |
| `message` | `role` (`system`, `user`, or `assistant`), `content` (part list) |
| `tool_call` | `name`, `arguments` (any JSON), optional `call_id` |
| `tool_result` | `output` (part list), optional `call_id` (the call it answers), optional `error` |
| `reasoning` | `text` |
| `custom` | `kind` (a name such as `acme:handoff`), `data` (any JSON) |

## Parts

| `type` | Fields |
| --- | --- |
| `text` | `text` |
| `image`, `audio`, `file` | `source`: `{"type": "url", "url": ...}` or `{"type": "base64", "data": ...}`; optional `media_type`, such as `image/png` |
| `json` | `value` (any JSON) |
| `custom` | `kind`, `data` (any JSON) |

Use `custom` items and parts for data observations do not define yet.

## Usage

Every field is optional. Leave out what you don't know; don't send zero.

| Field | Meaning |
| --- | --- |
| `non_cached_input_tokens` | Input tokens charged at the normal rate. |
| `non_cached_input_image_tokens` | Image input tokens charged at the normal rate. |
| `cache_read_input_tokens` | Input tokens read from the prompt cache. |
| `cache_creation_input_tokens` | Input tokens written to the prompt cache. |
| `output_tokens` | Generated tokens, excluding reasoning. |
| `output_image_tokens` | Generated image tokens. |
| `reasoning_tokens` | Tokens spent reasoning. |
| `total_cost_usd` | The call's cost in USD. |

Each field means the same as the matching `judgment.usage.*` attribute in [Attribute keys](/documentation/reference/attribute-keys-and-mappers#usage--cost).

## Errors

`error` on the observation records a failed model call. `error` on a `tool_result` records a failed tool. Both take optional `type`, `message`, and `stacktrace`; `{}` records a failure without details. Judgment never infers a failure from span status or a missing output.

## Rules

- **Send the whole window.** `input` holds every item the model was given, including history from earlier calls. Judgment stores each distinct item once per conversation, so resent history costs little storage. It still counts toward the span's size.
- **Keep per-call facts off items.** An item must be exactly what is resent on later calls. Finish reason, usage, and latency belong on the observation.
- **No unknown fields.** Every object is closed. A field this page does not define rejects the whole observation.

## Validation and limits

An observation that does not fit this format is not stored. Judgment sets `judgment.conversation.warnings` on its span, listing each problem by field path. The span and its trace are stored as usual.

| Limit | Value |
| --- | --- |
| IDs, names, kinds, models, finish reasons, media types | 1 to 512 characters |
| Span attributes, including the observation | 8 MiB per span. See [Ingest contract](/documentation/reference/ingest-contract). |