Task set v1.0 / Core tasks
ARTB-018 · v1.0
What changed in the latest release?
An app I use just updated. I want to know what changed, from the official source.
Research & dataData access
What we say to the agent
What's the latest version of software and what changed? Link the official release notes.
How it is judged
Pass
- Correct latest version, at least 2 correct changes, official link.
Partial
- Correct version, but changes or link incomplete.
Fail
- Outdated version or invented changes.
Changes every round
- {software}
Prompt templates are public. The concrete values are published with each round's results.
Evidence we keep
- screen_recording
- changelog_screenshot