Andreas Sotirakopoulos ; Kirstie Hawkey ; Konstantin Beznosov

28 June 2010

Abstract: We recently replicated and extended a 2009 study that investigated the effectiveness of SSL warnings. Our experimental design aimed to mitigate some of the limitations of that prior study, including allowing participants to use their web browser of choice and recruiting a more representative user sample. However, during this study we observed and measured a strong bias in participants’ behaviour due to the laboratory environment. In this paper we discuss the challenges of observing natural behaviour in a study environment, as well as the challenges of replicating previous studies, given the rapid changes in web technology. Finally, we propose alternatives to traditional laboratory study methodologies that can be considered by the usable security research community when investigating research questions involving sensitive data where trust may influence behaviour.

Keyword(s): Usable Security ; Experimental Design ; Study Environment Bias ; Study Replication

Published in: A. Sotirakopoulos, K. Hawkey, and K. Beznosov. "I did it because I trusted you": Challenges with the study environment biasing participant behaviours. In SOUPS Usable Security Experiment Reports (USER) Workshop, 2010.: