000000243 001__ 243
000000243 005__ 20130522141941.0
000000243 037__ $$aLERSSE-RefConfPaper-2010-005
000000243 100__ $$aPooya Jaferian
000000243 245__ $$aChallenges in evaluating complex IT security management systems
000000243 260__ $$c2010-07-08
000000243 300__ $$a6
000000243 520__ $$aPerforming ecologically valid user studies for IT security management (ITSM) systems is challenging. The users of these systems are security professionals who are difficult to recruit for interviews, let alone controlled user studies. Furthermore, evaluation of ITSM systems inherits the difficulties of studying collaborative and complex systems.  During our research, we have encountered many challenges in studying ITSM systems in their real context of use. This has resulted in us investigating how other usability evaluation methods could be viable components for identifying usability problems in ITSM tools. However, such methods need to be evaluated and proven to be effective before their use. This paper provides an overview of the challenges of performing controlled user studies for usability evaluation of ITSM systems and proposes heuristic evaluation as a component of usability evaluations of these tools. We also discuss our methodology for evaluating a new set of usability heuristics for ITSM and the unique challenges of running user studies for evaluating usability evaluation methods.
000000243 6531_ $$aUsable Security
000000243 6531_ $$aUser Studies
000000243 6531_ $$aHeuristic Evaluation 
000000243 6531_ $$aIdM 
000000243 700__ $$aKirstie Hawkey
000000243 700__ $$aKonstantin Beznosov
000000243 8560_ $$fpooya@ece.ubc.ca
000000243 8564_ $$uhttp://lersse-dl.ece.ubc.ca/record/243/files/243.pdf$$yTransfer from CDS 0.99.7
000000243 909C4 $$pP. Jaferian, K. Hawkey, and K. Beznosov. Challenges in evaluating complex IT security management systems. In SOUPS Usable Security Experiment Reports (USER) Workshop, 2010.
000000243 980__ $$aRefConfPaper