Catalog

3 evals — Safety / Security

The same catalog/evals.yaml the CLI reads. Live means it runs end-to-end today; building and roadmap entries show exactly what is coming and welcome contributions.

All Agent / Tool use Code Reasoning Embodied Robotics Safety / Securitylive building roadmap

Eval	Category	Paper	License	Status
HarmBench Red-teaming behaviors (standard+contextual), judged; score = 1 - ASR.	Safety / Security	Mazeika et al., 2024, arXiv:2402.04249	MIT	Live
AILuminate MLCommons hazard prompts (public practice set), judged safe-response rate.	Safety / Security	MLCommons, 2024, arXiv:2503.05731	CC-BY-4.0	Live
JailbreakBench Refusal robustness on the JBB harmful behaviors (no-attack baseline; judged).	Safety / Security	Chao et al., 2024, arXiv:2404.01318	MIT	Live