The Quick Association Check (QuACk): a resource-light, ‘bias robust’ method for exploring the relationship between mental models and behaviour patterns with home heating systems

ABSTRACT Easy-to-apply methods, allowing exploration of users’ mental models and behaviour with devices, are scarce. Existing methods can be resource-intensive, or fail to mitigate bias, threatening output validity. This paper describes the development of the Quick Association Check (QuACk) and highlights the methodological improvements following qualitative iterations. QuACk is a semi-structured interview with paper-based activities and templates, constructed with consideration of bias from the outset, offering a quick, resource light method to explore association between householders’ heating mental models and behaviour patterns. Through qualitative iterations of the method, key revisions identified the need to: (1) strongly emphasise positioning, guidance and structure; (2) re-categorise existing ‘shared models’ in the literature as ‘device generic’, rather than ‘thermostat specific’ during data interpretation and (3) consider an association at the system level, rather than the device level. The potential to apply the findings of QuACk to energy-reducing strategies is discussed, and generalisability considered.

[1]  Robert C. Williges,et al.  The Society's Lecture 1987 The Use of Models in Human-Computer Interface Design , 1987 .

[2]  Guy H. Walker,et al.  Human Factors Methods: A Practical Guide for Engineering and Design , 2012 .

[3]  Richard J. Jagacinski,et al.  Describing the Human Operator's Internal Model of a Dynamic System , 1978 .

[4]  Neville A. Stanton,et al.  On the reliability and validity of, and training in, ergonomics methods: a challenge revisited , 2016 .

[5]  Neville A. Stanton,et al.  The development of the Schema World Action Research Method (SWARM) for the elicitation of perceptual cycle data , 2016 .

[6]  Annika F L Larsson,et al.  Driver usage and understanding of adaptive cruise control. , 2012, Applied ergonomics.

[7]  Wiktoria Glad,et al.  Housing renovation and energy systems: the need for social learning , 2012 .

[8]  J. Mathieu,et al.  The influence of shared mental models on team process and performance. , 2000, The Journal of applied psychology.

[9]  Denis Besnard,et al.  Cognitive mismatches in the cockpit: will they ever be a thing of the past? , 2007, Applied ergonomics.

[10]  Nicola Combe,et al.  Assessing the number of users who are excluded by domestic heating controls , 2011 .

[11]  Dedre Gentner,et al.  Understanding Micronesian Navigation , 2014 .

[12]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[13]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[14]  Johan de Kleer and John Seely Brown,et al.  Assumptions and Ambiguities in Mechanistic Mental Models , 2014 .

[15]  Kirsten M. A. Revell,et al.  When energy saving advice leads to more, rather than less, consumption , 2017 .

[16]  Willett Kempton,et al.  Cultural models in language and thought: Two theories of home heat control , 1987 .

[17]  P. A. Hancock,et al.  On the relevance of qualitative methods for ergonomics , 2004 .

[18]  Mark S. Young,et al.  Drive-by-wire: The case of driver workload and reclaiming control with adaptive cruise control , 1997 .

[19]  Alan Jackson,et al.  Investigation into issues of passenger egress in Ladbroke Grove rail disaster. , 2005, Applied ergonomics.

[20]  John R. Wilson,et al.  Horses for courses – but no favourites. A reply to three commentaries , 2004 .

[21]  A. Wirth,et al.  Analyzing shared and team mental models , 2001 .

[22]  Willett Kempton,et al.  Two Theories of Home Heat Control , 1986, Cogn. Sci..

[23]  Neville A Stanton,et al.  Models of models: filtering and bias rings in depiction of knowledge structures and their implications for design , 2012, Ergonomics.

[24]  Peter M. Chisnall,et al.  Questionnaire Design, Interviewing and Attitude Measurement , 1993 .

[25]  S. Payne.  A descriptive study of mental models , 1991 .

[26]  Christopher D. Wickens,et al.  The Transfer of Failure-Detection Skills between Monitoring and Controlling Dynamic Systems , 1982 .

[27]  Christopher D. Wickens,et al.  Pilots' Monitoring Strategies and Performance on Automated Flight Decks: An Empirical Study Combining Behavioral and Eye-Tracking Data , 2007, Hum. Factors.

[28]  Neville A Stanton,et al.  Case studies of mental models in home heat control: searching for feedback, valve, timer and switch theories. , 2014, Applied ergonomics.

[29]  J. Greene,et al.  Adolescents' Use of Self-Regulatory Processes and Their Relation to Qualitative Mental Model Shifts While Using Hypermedia , 2006, ICLS.

[30]  Benjamin Naumann.  The Architecture Of Cognition , 2016 .

[31]  W. Brewer.  Schemas versus mental models in human memory. , 1987 .

[32]  Rocky Ross,et al.  Mental models , 2004, SIGA.

[33]  L. Desnoyers.  The role of qualitative methodology in ergonomics: a commentary , 2004 .

[34]  John Annett,et al.  A note on the validity and reliability of ergonomics methods , 2002 .

[35]  P.A. Hancock,et al.  Individuation: the N = 1 revolution , 2009 .

[36]  Mark S. Young,et al.  Guide to Methodology in Ergonomics: Designing for Human Use , 1999 .

[37]  D. Gentner,et al.  Cultural models in language and thought: How people construct mental models , 1987 .

[38]  David E. Kieras,et al.  The Role of a Mental Model in Learning to Operate a Device , 1990, Cogn. Sci..

[39]  Neville A Stanton,et al.  The famous five factors in teamwork: a case study of fratricide , 2010, Ergonomics.

[40]  Claudia R Binder,et al.  System Perspectives of Experts and Farmers Regarding the Role of Livelihood Assets in Risk Perception: Results from the Structured Mental Model Approach , 2009, Risk analysis : an official publication of the Society for Risk Analysis.

[41]  Therese Peffer,et al.  Facilitating energy savings with programmable thermostats: evaluation and guidelines for the thermostat user interface , 2013, Ergonomics.

[42]  R. Cole,et al.  Influence of occupants' knowledge on comfort expectations and behaviour , 2009 .

[43]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[44]  Linden J. Ball,et al.  Internal representations, external representations and ergonomics: towards a theoretical integration , 2009 .

[45]  Loren Lutzenhiser,et al.  The "Average American" Unmasked: Social Structure and Differences in Household Energy Use and Carbon Emissions , 2008 .

[46]  Michael I. Gentry,et al.  Central heating thermostat settings and timing: building demographics , 2010 .

[47]  John R. Wilson,et al.  Mental Models: Theory and Application in Human Factors , 1989 .

[48]  Guy H. Walker,et al.  Hierarchical task analysis vs. cognitive work analysis: comparison of theory, methodology and contribution to system design , 2010 .

[49]  John M. Carroll,et al.  Mental models in human-computer interaction: research issues about what the user of software knows , 1987 .

[50]  Mark S. Young,et al.  A proposed psychological model of driving automation , 2000 .

[51]  Craig Eidman,et al.  Using knowledge structures to redesign an instructor-operator station. , 2011, Applied ergonomics.

[52]  N Moray,et al.  Designing for transportation safety in the light of perception, attention, and mental models. , 1990, Ergonomics.

[53]  Donald A. Norman,et al.  Some observations on mental models , 1987 .

[54]  Nicolas Marmaras,et al.  Conventional vehicle display panels: the drivers' operative images and directions for their redesign. , 2012, Applied ergonomics.

[55]  J. Gerring.  A case study , 2011, Technology and Society.

[56]  D G Wastell,et al.  Designing for the home: a comparative study of support aids for central heating systems. , 2009, Applied ergonomics.

[57]  Bruno Rüttinger,et al.  Ecological performance of electrical consumer products: the influence of automation and information-based measures. , 2004, Applied ergonomics.

[58]  Kirsten Schreibers,et al.  Human-factors engineering for smart transport: design support for car drivers and train traffic controllers. , 2006, Applied ergonomics.

[59]  Neville A Stanton,et al.  Driver behaviour with adaptive cruise control , 2005, Ergonomics.

[60]  Neville A Stanton,et al.  Mind the gap - Deriving a compatible user mental model of the home heating system to encourage sustainable behaviour. , 2016, Applied ergonomics.

[61]  Robert A. Jacobs,et al.  Are People Successful at Learning Sequences of Actions on a Perceptual Matching Task? , 2011, Cogn. Sci..

[62]  H. Kanis,et al.  The quantitative-qualitative research dichotomy revisited , 2004 .

[63]  J. Rasmussen,et al.  Mental procedures in real-life tasks: a case study of electronic trouble shooting. , 1974, Ergonomics.

[64]  G. Grote,et al.  Adaptive coordination and heedfulness make better cockpit crews , 2010, Ergonomics.

[65]  D. Gentner,et al.  Flowing waters or teeming crowds: Mental models of electricity , 1982 .

[66]  Therese Peffer,et al.  How people use thermostats in homes: A review , 2011, Building and Environment.

[67]  Linden J. Ball,et al.  Analogical Reasoning and Mental Simulation in Design: Two Strategies Linked to Uncertainty Resolution , 2009 .

[68]  N. M. Morris,et al.  On Looking into the Black Box: Prospects and Limits in the Search for Mental Models , 1986 .

[69]  A. Tversky,et al.  Judgment under Uncertainty , 1982 .