Task set v1.0 / Local: US apps
ARTB-G01 · v1.0
Amazon vs Walmart vs Target
Same exact item, which store is cheapest today?
ShoppingPurchasingData access
What we say to the agent
Compare the price of product on Amazon, Walmart and Target right now. Give links.
How it is judged
Pass
- All 3 prices within 5% of the baseline, same SKU, working links, correct cheapest store.
Partial
- 2 of 3 prices correct.
Fail
- 1 or fewer correct, or an invented price.
Changes every round
- {product}
Prompt templates are public. The concrete values are published with each round's results.
Evidence we keep
- screen_recording
- baseline_screenshots