v0.4 Release Notes (Aug 1, 2025)
New Features
Enhanced Rules Engine
- PromptScorer Rules: Use your PromptScorers as metrics in automated rules, enabling rule-based actions triggered by your custom scoring logic. Configure rules with PromptScorers →
Access Control Enhancement
- New Viewer Role: Added a read-only role that provides access to view dashboards, traces, evaluation results, datasets, alerts, and other platform data without modification privileges - perfect for stakeholders who need visibility without editing access
Data Exporting
- Trace Export: Export selected traces from monitoring and dataset tables as JSONL files for external analysis or archival purposes. Export traces →
Improvements
Data Management
- Paginated Trace Fetching: Implemented efficient pagination for viewing large volumes of traces, making it faster to browse and analyze your monitoring data
- Multi-Select and Batch Operations: Select multiple tests and delete them in bulk for more efficient test management
Evaluation Expected Behavior
- Consistent Error Scoring: Custom scorers that encounter errors now automatically receive a score of 0, ensuring clear identification of failed evaluations in your data
Developer Experience
- Enhanced Logging: Added detailed logging for PromptScorer database operations to help debug and monitor scorer creation and updates
User Experience
- Enhanced Action Buttons: Improved selection action bars across all tables with clearer button styling, consistent labeling, and better visual hierarchy for actions like delete and export
- Streamlined API Key Setup: Copy API keys and organization IDs as pre-formatted environment variables (
JUDGMENT_API_KEYandJUDGMENT_ORG_ID) for faster application configuration