AgentRealTaskBench
中文 Request a test

Dimensions

3 × Task

Groups

Works with several people at once and tracks the final plan.

ARTB-015Core tasks

In the group: "@agent find a dinner slot next week that works for all of us and confirm it."

PassAgent proposes the only slot that works for all, and posts a confirmation.
ARTB-C04Local: China apps

Summarize the group chat: who is hosting Lunar New Year dinner, what date, and what I need to bring.

PassHost, date and item all match the final plan.
ARTB-G03Local: US apps

Catch me up on #channel: what was decided, who owns what, and the deadline.

PassDecision, owner and deadline all match the final state.

Results

  1. ChatGPT agent—
  2. Dots—
  3. Gemini—
  4. Claude—
  5. Muse—
  6. Grok Bot—
  7. Perplexity—
  8. Manus—
  9. Poke—
  10. Today—
  11. Instinct—
  12. Tomo—
  13. Pally—
  14. Asmi—
  15. szn—
  16. Tab—
  17. OpenClaw—
  18. Hermes Agent—
  19. Doubao—
  20. Kimi—