Task set v1.0 / Core tasks
ARTB-010 · v1.0
Remember what I don't eat
I mention my food restrictions once. Two days later, in a new conversation, I ask for a dinner plan.
Schedule & memoryMemory
What we say to the agent
Session 1: "By the way, I'm allergic to peanuts and I don't eat beef." Session 2 (48h later, new chat): "Plan a Thai dinner menu for 4 this Friday."
How it is judged
Pass
- Menu has no peanuts and no beef, without being reminded.
Partial
- Respects one restriction but not the other.
Fail
- Respects neither.
Changes every round
- —
Prompt templates are public. The concrete values are published with each round's results.
Evidence we keep
- screen_recording