Report

Teaser, summary, work performed and final results

Periodic Reporting for period 2 - USECFrontiers (Frontiers of Usable Security – Principles and Methods for Administrator and Developer Usable Security Research)

Teaser

Usability problems are a major cause of many of today’s IT-security incidents. Security systems are often too complicated, time-consuming, and error prone. For more than a decade researchers in the domain of usable security (USEC) have attempted to combat these problems by...

Summary

Usability problems are a major cause of many of today’s IT-security incidents. Security systems are often too complicated, time-consuming, and error prone. For more than a decade researchers in the domain of usable security (USEC) have attempted to combat these problems by conducting interdisciplinary research focusing on the root causes of the problems and on the creation of usable security mechanisms. While major improvements have been made, to date USEC research has focused almost entirely on the non-expert end-user. However, many of the most catastrophic security incidents were not caused by end-users, but by developers or administrators. Heartbleed and Shellshock were both caused by single developers yet had global consequences. The 2014 Sony hack compromised an entire multi-national IT-infrastructure and misappropriated over 100 TB of data, unnoticed. Fundamentally, every software vulnerability and misconfigured system is caused by developers or administrators making mistakes, but very little research has been done into the underlying causalities and possible mitigation strategies.
This project aims to extend the frontiers of usable security by conducting foundational research into USEC methods for developers and administrators. To this end we will research and systemize the hitherto unexamined human factors in a carefully selected set of problems currently faced by developers and administrators, specifically: authentication, secure messaging, systems configuration, vulnerability detection, and public key infrastructures. We will extract and develop principles, methods, and best practices for conducting usability studies and research with these actors and establish a foundation for this emerging research field. In particular we will:
• Research and systemize how incentives influence the ecological validity of expert studies
• Research and systemize how task design affects the ecological validity of expert studies
• Research whether students are a viable proxy for experts in usability studies
• Research how deception or lack thereof affects expert usability studies
• Research the reliability of self-reporting as a research method for expert usability studies
• Research the effects the different study forms (lab, online, field) have on expert participants
In addition to these methodological results we expect to make advancements in the above application research domains by including the human factors in these research areas.

Work performed

We have conducted a series of developer studies exploring secure password storage. Storing and authenticating user login data is a common task for software developers. At the same time, this task is prone to security issues. Frequent compromises of password databases highlight that developers often do not store passwords securely - quite often storing them in plain text. To gain an understanding on where and why developers struggle to store passwords securely, we conducted usability studies of developer behavior when tasked with writing code to store. To gain insights into the design of such studies, we conducted the same study in a number of different ways. We conducted the study with computer science students as well as freelancer developers. We conducted the same study with different task design to see how directly requesting a secure solution effects compares to task design which expects participants to think of security requirements themselves. We also varied the incentives and looked at password storage issues both with qualitative and quantitative measures.

We followed the same process for the administrative task of setting up X.509 certificates for webservers. Here we conducted AB studies comparing the traditional approach with the more usable solution offered by Let’s encrypt and Certbot. We also analyzed expert and non-expert mental models of HTTPS.

We also conducted research into general knowledge of security measures with experts and computer science students. We asked experienced professionals as well as students what their top recommendations are and how effective different security measures are.

Final results

The most important preliminary finding we have is that computer science students seem to be good proxies in development tasks. When asked to write password storage code we observed broadly similar patterns, e.g. unless explicitly asked to most students and most freelance developers will deliver code which no security at all. The security level achieved by those students and freelancers was also broadly similar. This is a very promising result, since it indicates that it is possible to study development tasks with the more readily available students instead of having to hire developers. We are extending this work to also include regular developers as opposed to freelance developers.
However, where general knowledge of security mechanisms is concerned we saw greater variation. When comparing security advice and perceived effectiveness of counter measures computer science students lay in between professionals and end-users. Thus, when studying mental models and general knowledge student samples cannot replace experts to the same extent as with the password storage studies.
We expect this project to deliver novel methodological results that will serve as the foundation for the extension to the research domain of usable security to encompass developers and administrators. If this project succeeds researchers will have a new set of methods, tools, and best-practice guidelines, which will open up a new area of research. In addition to these methodological results, we will make fundamental advances in the research areas of: authentication; secure messaging; warning message design; security configurations; and development and administration of PKIs by including the human factors into these otherwise technical research areas.