June 23, 2025

New Features

Multi-Agent Research Demo

• Added comprehensive multi-agent research demo to the judgeval package
• Four Interactive Examples: Shows users how to run multi-agent systems, trace parallel agents, create/edit documents with text and images, and export data for optimization (SFT, RL, etc.)
• Perfect for understanding multi-agent workflows and tracing capabilities

Automated Dataset Actions for Rules

• Rule-Triggered Dataset Addition: Automatically add traces to datasets when specific rules are triggered
• Smart Organization: For example, traces taking longer than 30 minutes can be automatically grouped into datasets for performance analysis
• Streamlines debugging workflows by providing clearer groupings of problematic traces

Trace Tagging and Search

• Flexible Tagging System: Apply custom labels to traces via SDK (add_metadata) or platform interface
• Platform-Wide Search: Press CMD/ALT + K to instantly search by project name, trace name, or tags
• Quick Discovery: Results ordered by creation time for easy browsing and faster navigation
• Dramatically reduces time spent locating relevant traces for debugging and analysis

Improvements

SDK Reliability

• Exponential Backoff: Added to all outgoing SDK HTTP requests for improved reliability and robustness of backend communication
• Enhanced OpenAI Support: wrap() now supports tracing OpenAI's client.beta.chat.completions.parse() method

Dataset Interface

• Simplified Dataset Usage: Streamlined interface for easier dataset management and interaction

Documentation Updates

• New documentation for automated dataset actions and rules
• Updated tagging and search functionality guides
• Added multi-agent demo tutorials and examples