---
title: Offline Test Runner
seo:
  title: Offline Test Runner — Python SDK
  description: >-
    Reference for JudgeVersionPin, OfflineTestRunner, normalize_judge_versions,
    and build_agent_kwargs. (Python SDK)
description: >-
  Reference for JudgeVersionPin, OfflineTestRunner, normalize_judge_versions,
  and build_agent_kwargs.
---

| Prop | Type | Default | Description |
| - | - | - | - |
| `AgentFunction?` | `Any` | `Callable[..., Any]` | |
| `PassConditionFn?` | `Any` | `Callable[[Dict[str, Any], List[ScorerData]], bool]` | |
| `TERMINAL_STATUSES?` | `Any` | `frozenset({'completed', 'error', 'cancelled'})` | |
| `EXAMPLES_PAGE_SIZE?` | `Any` | `100` | |
| `ITEMS_PAGE_SIZE?` | `Any` | `200` | |

**Classes**

**[JudgeVersionPin](/sdk-reference/python/offline_tests/offline_test_runner/JudgeVersionPin)**

**[OfflineTestRunner](/sdk-reference/python/offline_tests/offline_test_runner/OfflineTestRunner)**

**Functions**

## normalize\_judge\_versions()

Validate and normalize `judge_versions` entries.

Each entry must identify a judge by `name` (or `judge_id`) and may pin
a `tag`, `version`, or `major_version`/`minor_version` pair. Judges
not listed default to their `prod` tag (else latest) server-side.

**ValueError**: If an entry is not a dict or identifies no judge.

```python
def normalize_judge_versions(judge_versions) -> Optional[List[Dict[str, Any]]]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `judge_versions` | `Optional[List[JudgeVersionPin]]` | - | |

### Returns

`Optional[List[Dict[str, Any]]]`

***

## build\_agent\_kwargs()

Map an example's data fields onto the agent entrypoint's parameters.

Each declared parameter is filled from the example field of the same name,
or from a custom-mapped field via `field_mapping` (which maps each
parameter name to the dataset field it reads). Example fields the entrypoint
does not declare are
ignored -- so a dataset can carry extra columns (e.g. `trace`) the agent
doesn't use -- unless the entrypoint accepts `**kwargs`, in which case the
leftover fields are passed through too. The match succeeds as long as the
example supplies every required (no-default) parameter.

**TypeError**: only if a required parameter has no matching example field.

```python
def build_agent_kwargs(agent_function, data, field_mapping=None) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `agent_function` | `AgentFunction` | - | |
| `data` | `Dict[str, Any]` | - | |
| `field_mapping?` | `Optional[Dict[str, str]]` | `None` | |

### Returns

`Dict[str, Any]`

***

## \_parse\_reason()

Coerce a stored scorer reason into the `{text, citations?}` wire shape.

```python
def _parse_reason(raw) -> Dict[str, Any]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `raw` | `Any` | - | |

### Returns

`Dict[str, Any]`

***

## \_reason\_text()

```python
def _reason_text(raw) -> Optional[str]:
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `raw` | `Any` | - | |

### Returns

`Optional[str]`