Vercel AI SDK

Vercel AI SDK integration captures traces from your Vercel AI SDK applications, including agent execution flow, tool invocations, and multi-step reasoning processes. This integration is designed for TypeScript applications.

Refer to the Judgeval Tracer documentation for more information on how to instrument your application.

Quickstart

Install Dependencies

npm install ai @ai-sdk/openai judgeval zod dotenv mathjs

yarn add ai @ai-sdk/openai judgeval zod dotenv mathjs

pnpm add ai @ai-sdk/openai judgeval zod dotenv mathjs

bun add ai @ai-sdk/openai judgeval zod dotenv mathjs

Initialize Tracer

Create an instrumentation file to initialize the Judgment tracer:

instrumentation.ts

import { Judgeval } from "judgeval";

export const client = Judgeval.create();

export const tracer = await client.nodeTracer.create({
  projectName: "AI SDK Weather Agent",
});

Enable Telemetry

Enable the experimental_telemetry option in your AI SDK application and set isEnabled to true. This will emit OpenTelemetry data which can be automatically captured by Judgeval's tracer.

weatherAgent.ts

import { openai } from "@ai-sdk/openai";
import { generateText, tool } from "ai";
import { z } from "zod";
import { tracer } from "./instrumentation"; 

async function main() {
  const result = await generateText({
    model: openai("gpt-5"),
    tools: {
      weather: tool({
        description: "Get the weather in a location",
        inputSchema: z.object({
          location: z.string().describe("The location to get the weather for"),
        }),
        execute: async ({ location }) => ({
          location,
          temperature: 72 + Math.floor(Math.random() * 21) - 10,
        }),
      }),
    },
    experimental_telemetry: { 
      isEnabled: true, 
    }, 
    prompt: "What is the weather in San Francisco?",
  });

  console.log(`Result: ${JSON.stringify(result, null, 2)}`);

  return result;
}

await main().catch(console.error);
await tracer.shutdown();

If you are using Open Router as your model provider, make sure to enable OpenRouter Usage Accounting to enable cost tracking.

Example: Math Agent with Multi-Step Reasoning

instrumentation.ts

import { Judgeval } from "judgeval";

export const client = Judgeval.create();

export const tracer = await client.nodeTracer.create({
  projectName: "AI SDK Math Agent",
});

mathAgent.ts

import { openai } from "@ai-sdk/openai";
import { generateText, stepCountIs, tool } from "ai";
import * as mathjs from "mathjs";
import "dotenv/config";
import { z } from "zod";
import { tracer } from "./instrumentation";

async function main() {
  const { text: answer } = await generateText({
    model: openai("gpt-5"),
    tools: {
      calculate: tool({
        description:
          "A tool for evaluating mathematical expressions. Example expressions: " +
          "'1.2 * (2 + 4.5)', '12.7 cm to inch', 'sin(45 deg) ^ 2'.",
        inputSchema: z.object({ expression: z.string() }),
        execute: async ({ expression }) => mathjs.evaluate(expression),
      }),
    },
    stopWhen: stepCountIs(10),
    onStepFinish: async ({ toolResults }) => {
      console.log(`STEP RESULTS: ${JSON.stringify(toolResults, null, 2)}`);
    },
    experimental_telemetry: {
      isEnabled: true,
    },
    system:
      "You are solving math problems. " +
      "Reason step by step. " +
      "Use the calculator when necessary. " +
      "The calculator can only do simple additions, subtractions, multiplications, and divisions. " +
      "When you give the final answer, provide an explanation for how you got it.",
    prompt:
      "A taxi driver earns $9461 per 1-hour work. " +
      "If he works 12 hours a day and in 1 hour he uses 14-liters petrol with price $134 for 1-liter. " +
      "How much money does he earn in one day?",
  });

  console.log(`FINAL ANSWER: ${answer}`);
}

await tracer.observe(main)().catch(console.error);
await tracer.shutdown();

For Quick Scripts: The Tracer.shutdown() call is essential for short-lived scripts to ensure all traces are exported before the process exits. For long-running servers (e.g., Express, Next.js), this is not necessary as the tracer will export spans continuously throughout the application lifecycle.

If you need additional time for async operations to complete before shutdown:

Node.js / Express:

await main().catch(console.error);
await new Promise((resolve) => setTimeout(resolve, 10000)); // Give time for traces to export
await tracer.shutdown();

Bun:

await main().catch(console.error);
await Bun.sleep(10000); // Give time for traces to export
await tracer.shutdown();

Next Steps

OpenTelemetry Integration - Learn more about OpenTelemetry tracing with Judgment.
Agent Behavior Monitoring - Monitor your Vercel AI SDK agents in production with behavioral scoring.
Tracing Reference - Learn more about Judgment's tracing capabilities and advanced configuration.