The Influence of Participants' Personality on Quantitative and Qualitative Metrics in Usability Testing

We present the results of a usability study with 35 participants investigating the influence of personality on various metrics used in usability engineering. We conduct a task based usability test with a website integrating tasks of various difficulty and measure performance metrics like task completion rate and time on task. We also use standard questionnaire based usability metrics like the System Usability Scale. Furthermore, we gather qualitative data via open-ended questions and count the number of words as well as the mentions of positive and negative aspects. We measure personality using the well-known big five model, also often referred to as OCEAN model (openness, conscientiousness, extraversion, agreeableness, neuroticism) and three basic needs (need for influence and power, need for recognition and performance, need for security and tranquility). We analyze the relationship between personality and usability metrics via correlations and regression models. We identify multiple significant results and show that in our study the personality correlated with some of the usability metrics we inspected. Extraversion and the need for influence and power show the most and strongest correlations. Furthermore, we also show that regression models based on personality traits can explain up to 37% of the variance in usability metrics. The results have implications for the improvement of the selection process of usability test participants as well as for the interpretation of test results. We discuss these implications and give an outlook on further research in this area.