Pooya Jaferian ; Kirstie Hawkey ; Andreas Sotirakopoulos ; Maria Velez-Rojas ; Konstantin Beznosov

29 July 2013

Abstract: The usability of IT security management (ITSM) tools is hard to evaluate by regular methods, making heuristic evaluation attractive. In this paper, we explore how domain specific heuristics are created by examining prior research in the area of heuristic and guideline creation. We then describe our approach of creating usability heuristics for ITSM tools, which is based on guidelines for ITSM tools that are interpreted and abstracted with activity theory. With a between-subjects study, we compared the employment of the ITSM and Nielsen's heuristics for evaluation of a commercial identity management system. Participants who used the ITSM set found more problems categorized as severe than those who used Nielsen's. We analyzed several aspects of our heuristics including the performance of individual participants using the heuristic, the performance of individual heuristics, the similarity of our heuristics to Nielsen's, and the participants' opinion about the use of heuristics for evaluation of IT security tools. We then discuss the implications of our results on the use of ITSM and Nielsen's heuristics for usability evaluation of ITSM tools.

Keyword(s): Heuristic Evaluation ; IT Security ; Usable Security ; Computer Supported Cooperative Work ; Activity Theory ; Usability Evaluation ; Identity Management ; Access Management
Note: Preprint


Published in: Pooya Jaferian, Kirstie Hawkey, Andreas Sotirakopoulos, Maria Velez-Rojas, Konstantin Beznosov, Heuristics for Evaluating IT Security Management Tools, Accepted in Human–Computer Interaction, doi:10.1080/07370024.2013.819198.: