Overmind Studio
score agent traces against eval expectations and grade train
Overmind Studio!!!!!!!!!!!!!!!!!!!!!!!! — score agent traces against eval expectations and grade training-ready datasets
How to use Overmind Studio
- Paste agent run transcripts into the traces panel, or use the samples
- Declare expectations: contains phrases, regex patterns or ordered checkpoints
- Press Run evaluation to score every trace against every expectation
- Check the pass-rate panel to see which expectations your agent fails most
- Export the graded dataset as JSON for training or review
Features
- Eval expectations in three kinds: contains, regex and ordered checkpoints
- Per-trace scores with pass counts and per-expectation pass rates
- Dataset grading that flags training-ready runs above your threshold
- One-click JSON export of the scored, versioned dataset
- Live threshold slider to tune what counts as training-ready
Frequently asked questions
Is it free?
Yes, unlimited.
Login to use this free online tool. Create a free account to get started.
