---
title: "LiteLLM Integration"
sidebar:
  label: "LiteLLM"
seo:
  title: "LiteLLM Integration with Judgment | Model Provider Docs"
  description: "Use Judgment with LiteLLM to capture model calls, trace agent behavior, and evaluate production AI systems."
---

{/* @test-flow
  id: provider-litellm
  lang: python
  env: JUDGMENT_API_KEY, JUDGMENT_ORG_ID, OPENAI_API_KEY
*/}

If you use [LiteLLM](https://github.com/BerriAI/litellm) within your application, you can trace, monitor, and analyze all LLM calls with **Judgment**. LiteLLM provides a unified interface to call 100+ LLM providers using a consistent API.

Because LiteLLM exposes a functional API (`litellm.completion`), the standard `wrap()` approach does not apply. Instead, register an OpenTelemetry instrumentor to automatically capture every LiteLLM call — including model name, token usage, and cost.

1. ### Install Dependencies

    **uv**

    ```bash
    uv add judgeval opentelemetry-instrumentation-litellm litellm
    ```

    **pip**

    ```bash
    pip install judgeval opentelemetry-instrumentation-litellm litellm
    ```

2. ### Initialize Tracing

    ```python title="setup.py"
    from judgeval import Tracer
    from opentelemetry.instrumentation.litellm import LiteLLMInstrumentor

    Tracer.init(project_name="litellm_project")
    Tracer.registerOTELInstrumentation(LiteLLMInstrumentor())
    ```

    `LiteLLMInstrumentor` monkeypatches `litellm.completion` and related functions so every call emits an OTEL span with the model name, prompt/completion tokens, and cost.

3. ### Use LiteLLM as Normal

    ```python title="app.py"
    import litellm
    from judgeval import Tracer
    from opentelemetry.instrumentation.litellm import LiteLLMInstrumentor

    Tracer.init(project_name="litellm_project")
    Tracer.registerOTELInstrumentation(LiteLLMInstrumentor())

    response = litellm.completion(
        model="gpt-4o",
        messages=[{"role": "user", "content": "Hello, world!"}]
    )

    print(response.choices[0].message.content)
    ```

    All LiteLLM calls are automatically traced and exported to the Judgment platform.

## Multi-Agent / Swarm Use Case

LiteLLM is commonly used inside multi-agent frameworks (e.g. swarm orchestrators) where each sub-agent calls `litellm.completion` directly. The instrumentor ensures that **cost is attributed to the spans where inference actually happens** — not the parent orchestrator.

```python title="swarm_example.py"
from judgeval import Tracer
from opentelemetry.instrumentation.litellm import LiteLLMInstrumentor

Tracer.init(project_name="swarm_project")
Tracer.registerOTELInstrumentation(LiteLLMInstrumentor())

@Tracer.observe(span_type="agent")
def data_worker(query: str) -> str:
    response = litellm.completion(
        model="gpt-4o",
        messages=[{"role": "user", "content": f"Process this data: {query}"}]
    )
    return response.choices[0].message.content

@Tracer.observe(span_type="agent")
def review_worker(data: str) -> str:
    response = litellm.completion(
        model="gpt-4o",
        messages=[{"role": "user", "content": f"Review this output: {data}"}]
    )
    return response.choices[0].message.content

@Tracer.observe(span_type="agent")
def orchestrator(query: str) -> str:
    data = data_worker(query)
    review = review_worker(data)
    return review

result = orchestrator("Analyze Q4 revenue trends")
```

In the Judgment trace view, each `data_worker` and `review_worker` span carries its own cost (from the underlying `litellm.completion` call), while the `orchestrator` span shows $0.00 since it makes no direct model calls.