Plug in.
Run the
eval.
Every major AGI benchmark behind two clean protocols. Bring any model endpoint and score it for real — no directory pretending to be a platform.
Two protocols
Everything hangs off two interfaces. An eval never imports an adapter; an adapter never imports an eval. Get these right and adding eval #2 through #50 is incremental work.
A model endpoint. Takes a prompt plus any eval-specific scenario, returns a response. Ships for every major backend.
def respond(self, request:
PatientRequest) -> PatientResponse: ...An eval. Takes a patient and a case, returns a scored result with a typed failure tag. The harness handles concurrency and errors.
def run_case(self, patient, case:
Case) -> ScoredResult: ...Live evals
| Eval | Category | Grading | Status |
|---|---|---|---|
| GPQA Diamond | Reasoning | mcq | Live |
| MMLU-Pro | Reasoning | mcq | Live |
| MATH | Reasoning | math | Live |
| AIME 2024 | Reasoning | math | Live |
| BIG-Bench Hard | Reasoning | exact-match | Live |
| MuSR | Reasoning | mcq | Live |
| ZebraLogic | Reasoning | logic | Live |
| HumanEval+ | Code | code | Live |
| LiveCodeBench | Code | code | Live |
| BigCodeBench | Code | code | Live |
| RepoBench | Code | code | Live |
| SWE-Lancer | Code | code | Live |
| τ-bench | Agent / Tool use | tool-use | Live |
| SWE-bench Verified | Agent / Tool use | agent | Live |
| SWE-bench Lite | Agent / Tool use | agent | Live |
| GAIA | Agent / Tool use | agent | Live |
| Berkeley Function-Calling Leaderboard | Agent / Tool use | function-calling | Live |
| ALFWorld | Embodied | text-game | Live |
| ScienceWorld | Embodied | text-game | Live |
| WebShop | Embodied | embodied | Live |
| LIBERO | Robotics | robotics | Live |
| HarmBench | Safety / Security | safety | Live |
| AILuminate | Safety / Security | safety | Live |
| JailbreakBench | Safety / Security | safety | Live |
Six categories
Climb it. Push it.
Run an eval, push your score, watch it over time — then forward your scoreboard to a larger challenge in one click.
| # | Model | Org | Eval | Score |
|---|---|---|---|---|
| 01 | claude-opus-4.8 | anthropic | gpqa-diamond | 0.811 |
| 02 | gpt-x | openai | gpqa-diamond | 0.774 |
| 03 | grok-4 | xai | gpqa-diamond | 0.738 |
| 04 | llama-4-405b | meta | humaneval-plus | 0.901 |
| 05 | qwen3-72b | alibaba | math | 0.689 |
Eight adapters
Benchmarks are research.
We credit it like research.
Every eval on this platform is years of someone's work, and scores tend to travel without the names attached. So crediting is built in: citations at the point of use, one-click BibTeX, verified links to star the upstream repo, and public run counts that show authors their work mattering.
How we credit researchers →