Catalog

2 evals — Code

The same catalog/evals.yaml the CLI reads. Live means it runs end-to-end today; building and roadmap entries show exactly what is coming and welcome contributions.

All Agent / Tool use Code Reasoning Embodied Robotics Safety / Securitylive building roadmap

Eval	Category	Paper	License	Status
HumanEval+ HumanEval with 80x more tests to catch incorrect-but-plausible solutions.	Code	Liu et al., 2023, arXiv:2305.01210	Apache-2.0	Live
LiveCodeBench Continuously updated contest problems to avoid training-set contamination.	Code	Jain et al., 2024, arXiv:2403.07974	CC-BY-4.0	Live