ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental Synthesis

The article introduces ScaleWoB, a framework for synthesizing high-fidelity interactive environments for GUI agents, enabling effective evaluation and training without the complexities of real-world environments. ScaleWoB supports over 100 environments and 1000 verifiable tasks, including a benchmark of 120 challenging tasks across 63 mobile applications, revealing that current mobile GUI agents have an average success rate of only 27.92% compared to 92.08% for humans. This framework's low resource requirements and ease of setup make it a significant advancement for practitioners developing and evaluating GUI agents in large-scale settings.