A Multi-Method Usability Evaluation of a Higher Education Service Evaluation System: Integrating Cognitive Walkthrough, Unmoderated Remote Testing, and Heuristic Evaluation

Abstract

This study evaluates the usability of a higher education service evaluation system using a multi-method approach that integrates Cognitive Walkthrough, Unmoderated Remote Usability Testing, and Heuristic Evaluation. The study was designed to diagnose usability barriers experienced by new and non-regular users when completing service evaluation tasks independently. The evaluation involved 15 actual users consisting of students, lecturers, and education staff, and three expert evaluators with knowledge of usability, Human-Computer Interaction, and interface design. Cognitive Walkthrough was used to examine learnability barriers across the task flow, Unmoderated Remote Usability Testing was used to capture user-performance evidence, and Heuristic Evaluation was used to classify problems and determine severity. The results show that the system can support completion of service evaluation tasks, but several critical usability issues remain. The most important problems are unclear submission confirmation, less intuitive menu labels, ambiguous service categories, long form interaction, weak mandatory-field cues, and low visibility of the submit button. Remote testing showed the lowest success rates in finding the evaluation menu (73.30%) and confirming successful submission (76.70%). Heuristic Evaluation rated weak submission confirmation as the highest-severity issue. This study contributes a triangulated usability evaluation model for higher education service evaluation systems and provides design recommendations to improve learnability, task efficiency, error prevention, user confidence, and the quality of institutional service evaluation data.