Threats to Reliability and Validity of Mixed Methods Research in User eXperience

The purpose of this paper is to shed some light on threats to reliability and validity of mixed methods (MM) research in user experience (UX) studies wherein quantitative and qualitative approaches are combined in data collection, analysis, and interpretation stages. The paper presents an empirical study of user experience where an integrated data analysis indicates a wide range of threats to the validity and reliability of mixed methods research. The authors also introduce and clarify how the data reduction technique and the typology of answers described in this paper could help to identify and address the quality shortcomings of mixed methods research in user experience and how the collected qualitative and quantitative data can supplement each other in the process. Given the early stage, exploratory nature and limited scope of the empirical study, the authors stop short from offering any prescriptive measures to those conducting or evaluating empirical mixed methods research in user experience, let alone other domains.