Automated Usability Evaluation of Graphical User Interfaces as well as Augmented and Virtual Reality

The usability of a software depends on its concrete application context, which is comprised of user groups, the tasks executed with the software, and the physical enviroment in which a software is used. To engineer software with a high usability, developers analyze the application context and design the software in a way that best supports its usage in the application context. However, users mostly use software in their own specific way and sometimes even for tasks the software is not designed for. In addition, the requirements towards a software may change, which may lead to new usage scenarios. Therefore, the initially good usability of the software may decrease. To address this issue, a continuous assessment of the usability of a software is required. As manual methods for usability assessments are cost intensive, we aim at developing fully automated usability evaluation methods.
In this research, we intersect possibilities for automated usability evaluation of user interfaces. This covers established Graphical User Interfaces (GUIs) as well as the more and more upcoming Augmented and Virtual Realities. To achieve our goals, we trace the actual usage of a software by recordings the actions of users. Based on the recordings and a resembled model of the user interface, we automatically generate task trees which are a model for the actual usage of the software. We use the task trees in combination with usage statistics and the user interface model to perform an automated usability evaluation of the software. The results of our approach provide detailed information about the detected usability issues including the performed user tasks and a proposal for solving the issues.