Andreas Sotirakopoulos ; Kirstie Hawkey ; Konstantin Beznosov

15 June 2011

Abstract: We replicated and extended a 2008 study conducted at CMU that investigated the e effectiveness of SSL warnings. We adjusted the experimental design to mitigate some of the limitations of that prior study; adjustments include allowing participants to use their web browser of choice and recruiting a more representative user sample. However, during our study we observed a strong disparity between our participants actions during the laboratory tasks and their self-reported "would be" actions during similar tasks in every day computer practices. Our participants attributed this disparity to the laboratory environment and the security it offered.In this paper we discuss our results and how the introduced changes to the initial study design may have affected them.Also, we discuss the challenges of observing natural behavior in a study environment, as well as the challenges of replicating previous studies given the rapid changes in web technology. We also propose alternatives to traditional laboratory study methodologies that can be considered by the usable security research community when investigating research questions involving sensitive data where trust may influence behavior.


Published in: Andreas Sotirakopoulos, Kirstie Hawkey, and Konstantin Beznosov. On the Challenges in Usable Security Lab Studies: Lessons Learned from Replicating a Study on SSL Warnings. In Proceedings of Symposium on Usable Privacy and Security, July 2011: