---
title: Judgeval
seo:
  title: Judgeval — Python SDK
  description: >-
    The main entry point for interacting with the Judgment platform. (Python
    SDK)
description: The main entry point for interacting with the Judgment platform.
---

`Judgeval` connects to your Judgment project and gives you access to
**SQL queries**, **evaluations**, and **datasets**.

Credentials are resolved in order: explicit arguments first, then
environment variables `JUDGMENT_API_KEY`, `JUDGMENT_ORG_ID`, and
`JUDGMENT_API_URL`.

**ValueError**: If any required credential or `project_name` is missing.

Minimal setup (credentials from environment variables):

```python
from judgeval import Judgeval

client = Judgeval(project_name="search-assistant")
```

Explicit credentials:

```python
client = Judgeval(
    project_name="search-assistant",
    api_key="jdg_...",
    organization_id="org_...",
)
```

Once initialized, use the `evaluation` and `datasets`
properties:

```python
eval_runner = client.evaluation.create()
dataset = client.datasets.get(name="golden-set")
```

## Attributes

| Prop | Type | Default | Description |
| - | - | - | - |
| `evaluation?` | `Any` | - | Access evaluations for scoring examples with hosted or custom judges. Use .create() to get an Evaluation you can call .run() on. eval_runner = client.evaluation.create() results = eval_runner.run( examples=examples, scorers=["faithfulness", "answer_relevancy"], eval_run_name="nightly-eval", ) |
| `offline_tests?` | `Any` | - | Access offline tests: test configs and dataset-backed test runs. Use .create_config() to bind a dataset to judges, and .run() to execute a test run (optionally driving an agent entrypoint and asserting a pass condition). config = client.offline_tests.create_config( name="nightly-regression", dataset="golden-set", judges=["helpfulness"], ) result = client.offline_tests.run( test_config="nightly-regression", agent_function=my_agent, pass_condition_fn=lambda fields, scorers: all( s.error is None for s in scorers ), assert_test=True, ) |
| `datasets?` | `Any` | - | Manage datasets of evaluation examples. Use .create(), .get(), or .list() to work with datasets. dataset = client.datasets.create( name="golden-set", schema={ "type": "object", "properties": { "input": {"type": "string"}, "expected_output": {"type": "string"}, }, }, examples=[ Example.create(input="What is 2+2?", expected_output="4"), ], ) |
| `prompts?` | `Any` | - | Manage versioned prompt templates with tagging support. Use .create(), .get(), .tag(), or .list() to work with prompts. prompt = client.prompts.create( name="system-prompt", prompt="You are a helpful assistant for {{product}}.", tags=["v1"], ) compiled = prompt.compile(product="Acme Search") |
| `agent_judges?` | `Any` | - | Manage Agent Judges (prompt-based scorers) on the platform. Use .create() or .update() to create and update prompt-based Agent Judges. judge = client.agent_judges.create( name="helpfulness", prompt="Score the assistant's helpfulness from 0 to 1.", model="gpt-5.2", score_type="numeric", ) client.agent_judges.update( judge_id=judge.judge_id, prompt="Updated rubric prompt.", ) |

***

## query()

Run a legacy JQL query, optionally narrowed by trace or session IDs.

**Deprecated.** Use [`sql()`](#sql) for new integrations, with SQL
predicates to narrow results. Existing JQL calls remain supported.

```python
def query(query, *, limit=None, trace_ids=None, session_ids=None) -> 'JqlQueryResponse':
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `query` | `'QueryInput'` | - | |
| `limit?` | `Optional[int]` | `None` | |
| `trace_ids?` | `Optional[Sequence[str]]` | `None` | |
| `session_ids?` | `Optional[Sequence[str]]` | `None` | |

### Returns

`'JqlQueryResponse'`

***

## discover\_schema()

Return the SQL schema reference as Markdown.

Mirrors MCP `discover_schema`: published tables, column types and
descriptions, row semantics, examples, and query limits. Fetches the
server's generated catalog using this client's credentials. Contains
no project data and does not require a resolved project or public
query opt-in.

```python
print(client.discover_schema())
```

```python
def discover_schema() -> str:
```

### Returns

`str` - The virtual schema reference as a Markdown string.

***

## sql()

Run one read-only SQL SELECT for this organization and project.

Prefer this method for new read-only queries. The server derives tenant
scope from the client's credentials and resolved project.
Call `discover_schema()` for supported tables and columns. Requires
viewer access and public SDK/API queries enabled for the organization.

Results are capped by the server at 1,000 rows and 5 MiB; exceeding
either cap returns an error. Use SQL predicates and LIMIT to narrow
results.

```python
result = client.sql("SELECT count() AS run_count FROM telemetry.traces")
print(result["rows"])
```

```python
def sql(sql_text) -> 'SqlResponse':
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `sql_text` | `str` | - | One SELECT against the virtual schema, at most 50,000 characters. Use SQL predicates to narrow the results. |

### Returns

`'SqlResponse'` - A dictionary with `catalog_version`, `columns` (name, type, nullable),
`rows` (dictionaries keyed by column name), `row_count`, and
`elapsed_ms`. Integers outside JavaScript's safe range arrive as
exact decimal strings.

***

## present()

Run a legacy JQL chart or table query.

**Deprecated.** Use [`sql()`](#sql) for new queries and render its
rows as charts or tables in your application. SQL does not return a
JQL presentation frame. Existing presentation calls and their frame
responses remain supported.

```python
def present(query, *, limit=None, trace_ids=None, session_ids=None) -> 'JqlPresentationResponse':
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `query` | `'QueryInput'` | - | |
| `limit?` | `Optional[int]` | `None` | |
| `trace_ids?` | `Optional[Sequence[str]]` | `None` | |
| `session_ids?` | `Optional[Sequence[str]]` | `None` | |

### Returns

`'JqlPresentationResponse'`

***

## discover()

Discover project-scoped judges, fields, models, and related values.

**Deprecated.** Use [`discover_schema()`](#discover_schema) to inspect
the SQL tables and columns, then [`sql()`](#sql) to query project values.
Schema discovery returns documentation, not project data. Existing
JQL discovery calls remain supported; SQL returns a different row schema.

```python
def discover(kind, *, limit=None, trace_ids=None, session_ids=None, **options) -> 'JqlQueryResponse':
```

### Parameters

| Prop | Type | Default | Description |
| - | - | - | - |
| `kind` | `'DiscoveryKind'` | - | |
| `limit?` | `Optional[int]` | `None` | |
| `trace_ids?` | `Optional[Sequence[str]]` | `None` | |
| `session_ids?` | `Optional[Sequence[str]]` | `None` | |
| `options?` | `Any` | `{}` | |

### Returns

`'JqlQueryResponse'`