Catalog

5 evals — Embodied

The same catalog/evals.yaml the CLI reads. Live means it runs end-to-end today; building and roadmap entries show exactly what is coming and welcome contributions.

All Agent / Tool use Code Reasoning Embodied Robotics Safety / Securitylive building roadmap

Eval	Category	Paper	License	Status
ALFWorld Household tasks in the real TextWorld engine; success rate over 134 unseen games.	Embodied	Shridhar et al., 2020, arXiv:2010.03768	MIT	Live
ScienceWorld Interactive science experiments requiring procedural understanding.	Embodied	Wang et al., 2022, arXiv:2203.07540	Apache-2.0	Live
WebShop Buy the right product from 1.18M items given a natural-language goal.	Embodied	Yao et al., 2022, arXiv:2207.01206	MIT	Live
Habitat ObjectNav Navigate photorealistic 3D homes to find an instance of a target object.	Embodied	Szot et al., 2021, arXiv:2106.14405	MIT	Roadmap
ALFRED Follow language instructions to complete household tasks from vision.	Embodied	Shridhar et al., 2019, arXiv:1912.01734	MIT	Roadmap