• Mapping the Design Space of User Experience for Computer Use Agents Mapping the Design Space of User Experience for Computer Use Agents AuthorsRuijia Cheng, Jenny T. • Liangâ , Eldon Schoop, Jeffrey Nichols View publication Copy Bibtex Large language model (LLM)-based computer use agents execute user commands by interacting with available UI elements, but little is known about how users want to interact with these agents or what design factors matter for their user experience (UX). • We conducted a two-phase study to map the UX design space for computer use agents. • In Phase 1, we reviewed existing systems to develop a taxonomy of UX considerations, then refined it through interviews with eight UX and AI practitioners. • The resulting taxonomy included categories such as user prompts, explainability, user control, and usersâ mental models, with corresponding subcategories and example design features. • In Phase 2, we ran a Wizard-of-Oz study with 20 participants, where a researcher acted as a web-based computer use agent and probed user reactions during normal, error-prone and risky execution.
Article Summaries:
- A recent study titled “Mapping the Design Space of User Experience for Computer Use Agents” explores how users interact with AI‑powered agents that perform tasks on their computers. Researchers first reviewed existing systems and interviewed eight UX and AI experts to create a taxonomy of UX factors-including user prompts, explainability, control, and mental models. In a second phase, a Wizard‑of‑Oz experiment with 20 participants tested a web‑based agent under normal, error‑prone, and risky conditions, validating the taxonomy and revealing how different design elements affect user needs and scenarios. The resulting framework offers developers a comprehensive map for designing more user‑friendly computer‑use agents.
Sources: