← Docs/ Agent / Tool use

API-Bank

Roadmap

Plan-and-call evaluation over a graded pool of tool APIs.

On the roadmap

API-Bank is catalogued but not runnable yet, so there are no usage docs — we do not document what does not run. The fact sheet below is sourced from the paper; the protocols it will implement are stable today.

Paper: API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs
Citation: Li et al., 2023, arXiv:2304.08244
License: MIT
Homepage: https://github.com/AlibabaResearch/DAMO-ConvAI/tree/main/api-bank

How an eval goes live

Implement an EvalRunner against the stable protocols.
Bundle a small real-schema sample so it runs offline.
Point the catalog entry's runner at the class.
Ship its docs in the same change — required to flip live.

pip install agi-evals