The tortoise and the (soft)ware: Moore's law, Amdahl's law, and performance trends for human-machine systems

Human interaction with computing and communication systems involves a mix of parallel and serial processing by the human-computer system. Moore's Law provides an illustration of the fact that the performance of the digital components of any human-computer system has improved rapidly. But what of the performance of those human components? While we humans are amazing information processing machines, our information processing capabilities are relatively fixed. This paper reviews 100 years of the human performance literature and shows, graphically, the disparity between the non-growth in human performance and the geometrical improvements in computational capability. Further, Amdahl's Law demonstrates, algebraically, that increasingly the (non-parallelizable) human performance becomes the determining factor of speed and success in most any human-computer system. Whereas engineered products improve daily, and the amount of information for us to potentially process is growing at an ever quickening pace, the fundamental building blocks of human-information processing (e.g., reaction time, short-term memory capacity) have the same speed and capacity as they did for our grandparents. Or, likely, for the ancient Greeks. This implies much for human-computer interaction design; rather than hoping our users to read or to type faster, we must look for optimally chosen human channels and maximally matched human and machine functions. This tortoise and the (hard-and soft-)ware race demands renewed enthusiasm for, and increased, systematic attention paid to the practice of usability and to research in human-computer interaction.

[1]  S Kornblum,et al.  The effects of stimulus-response mapping and irrelevant stimulus-response and stimulus-stimulus overlap in four-choice Stroop tasks with single-carrier stimuli. , 1998, Journal of experimental psychology. Human perception and performance.

[2]  Janan Al-Awar Smither,et al.  A Comparison of Broad Versus Deep Auditory Menu Structures , 2008, Hum. Factors.

[3]  Douglas J. Gillan,et al.  The Interface between Human Factors and Design , 1992 .

[4]  Randolph G. Bias,et al.  Cost-Justifying Usability: An Update for the Internet Age , 2005 .

[5]  D J Evans,et al.  Parallel processing , 1986 .

[6]  D. Mewhort,et al.  Analysis of Response Time Distributions: An Example Using the Stroop Task , 1991 .

[7]  D. Raab,et al.  Effect of stimulus-duration on auditory reaction-time. , 1962, The American journal of psychology.

[8]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[9]  C H BECK,et al.  Paced and self-paced serial simple reaction time. , 1963, Canadian journal of psychology.

[10]  Marian Petre,et al.  Usability Analysis of Visual Programming Environments: A 'Cognitive Dimensions' Framework , 1996, J. Vis. Lang. Comput..

[11]  David Sharek,et al.  A Useable, Online NASA-TLX Tool , 2011 .

[12]  Erik Brynjolfsson,et al.  The Productivity Paradox of Information Technology: Review and Assessment , 2014 .

[13]  Duncan R. Babbage COGNITIVE BARRIERS TO MAINSTREAM MOBILE COMPUTING DEVICES IN NEUROREHABILITATION , 2011 .

[14]  Rachel K. E. Bellamy,et al.  Deploying CogTool: integrating quantitative usability assessment into real-world software development , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[15]  W. Schilling,et al.  The Effect of Caffein and Acetanilid on Simple Reaction Time. , 1921 .

[16]  Simon Parsons,et al.  Principles of Data Mining by David J. Hand, Heikki Mannila and Padhraic Smyth, MIT Press, 546 pp., £34.50, ISBN 0-262-08290-X , 2004, The Knowledge Engineering Review.

[17]  M.H. Kryder,et al.  After Hard Drives—What Comes Next? , 2009, IEEE Transactions on Magnetics.

[18]  P. A. Vroon,et al.  Tapping rate and expectancy in simple reaction time tasks , 1973 .

[19]  Cathleen Wharton,et al.  Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces , 1990, CHI '90.

[20]  W. Rogers,et al.  Mechanisms underlying reduction in Stroop interference with practice for young and old adults. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[21]  Arthur I. Gates,et al.  The mnemonic span for visual and auditory digits. , 1916 .

[22]  J R Simon,et al.  Effects of S-R mapping and response modality on performance in a Stroop Task. , 1979, Journal of experimental psychology. Human perception and performance.

[23]  Ji Hyun active st century Park Flow in multitasking : the effects of motivation, artifact, and task factors , 2014 .

[24]  C S Green,et al.  Action-Video-Game Experience Alters the Spatial Resolution of Vision , 2007, Psychological science.

[25]  Stuart K. Card,et al.  Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a CRT , 1987 .

[26]  Gregory Piatetsky-Shapiro,et al.  Knowledge Discovery in Databases: An Overview , 1992, AI Mag..

[27]  P. J. Fay The effect of cigarette smoking on simple and choice reaction time to colored lights , 1936 .

[28]  Panagiotis G. Ipeirotis,et al.  Running Experiments on Amazon Mechanical Turk , 2010, Judgment and Decision Making.

[29]  L R Squire,et al.  On the relationship between recall and recognition memory. , 1992, Journal of experimental psychology. Learning, memory, and cognition.

[30]  I. Scott MacKenzie,et al.  Predicting text entry speed on mobile phones , 2000, CHI.

[31]  Thomas K. Landauer,et al.  The trouble with computers , 1995 .

[32]  Alexander Dekhtyar,et al.  Information Retrieval , 2018, Lecture Notes in Computer Science.

[33]  Alan D Baddeley,et al.  The complexities of complex span: explaining individual differences in working memory in children and adults. , 2003, Journal of experimental psychology. General.

[34]  Scott Isensee,et al.  User-Centered Design: An Integrated Approach with Cdrom , 2001 .

[35]  E. J. George,et al.  Foreperiod effect on time estimation and simple reaction time , 1977 .

[36]  D L Hintzman,et al.  "Stroop" effect: input or output phenomenon? , 1972, Journal of experimental psychology.

[37]  Nobuyuki Inui,et al.  Simple Reaction Times and Timing of Serial Reactions of Middle-Aged and Old Men , 1997, Perceptual and motor skills.

[38]  Carson Reynolds As we may communicate , 1998, SGCH.

[39]  Maryanne Martin,et al.  Memory span as a measure of individual differences in memory capacity , 1978 .

[40]  F. Donders On the speed of mental processes. , 1969, Acta psychologica.

[41]  Claudia Wallis,et al.  The multitasking generation. , 2006, Time.

[42]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[43]  J. Stroop Studies of interference in serial verbal reactions. , 1992 .

[44]  Allen Newell,et al.  The Prospects for Psychological Science in Human-Computer Interaction , 1985, Hum. Comput. Interact..

[45]  Heikki Mannila,et al.  Principles of Data Mining , 2001, Undergraduate Topics in Computer Science.

[46]  Geoffrey Underwood,et al.  Memory Span for Arabic Numerals and Digit Words: Evidence for a Limited-capacity, Visuo-spatial Storage System , 1999 .

[47]  A Henik,et al.  Interpreting Stroop interference: an analysis of differences between task versions. , 2001, Neuropsychology.

[48]  Randolph G. Bias,et al.  Cost-justifying usability , 2005 .

[49]  D L Kohfeld,et al.  Effects of the intensity of auditory and visual ready signals on simple reaction time. , 1969, Journal of experimental psychology.

[50]  A. Sanders Attention and performance III : proceedings of a symposium on attention and performance, held in Soesterberg August 4-8,1969, under the auspices of the Institute for Perception RVO-TNO, Soesterberg, the Netherlands , 1970 .

[51]  M. J. Emerson,et al.  The Unity and Diversity of Executive Functions and Their Contributions to Complex “Frontal Lobe” Tasks: A Latent Variable Analysis , 2000, Cognitive Psychology.

[52]  Alberta Steinman,et al.  Simple reaction time to change as a substitute for the disjunctive reaction , 1944 .

[53]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[54]  Tim Comber,et al.  Building usable web pages: an HCI perspective , 1995 .

[55]  Mary Beth Rosson,et al.  Getting around the task-artifact cycle: how to make claims and design by scenario , 1992, TOIS.

[56]  Roger Frost,et al.  International Organization for Standardization (ISO) , 2004 .

[57]  C. S. Green,et al.  Action video game modifies visual selective attention , 2003, Nature.

[58]  T. Hess Memory and aging in context. , 2005, Psychological bulletin.

[59]  Ian Witten,et al.  Data Mining , 2000 .

[60]  Paul W. Burgess,et al.  Real-world multitasking from a cognitive neuroscience perspective , 2000 .

[61]  Susan T. Dumais,et al.  Improving Web Search Ranking by Incorporating User Behavior Information , 2019, SIGIR Forum.

[62]  S. Dumais,et al.  Handbook of applied cognition , 2007 .

[63]  L Henderson,et al.  On the Peculiarity of Simple Reaction Time , 1990, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[64]  Austin Henderson,et al.  Usability of interactive systems: it will get worse before it gets better , 2012 .

[65]  B. C. Wright,et al.  Adults' versus children's performance on the Stroop task: interference and facilitation. , 2003, British journal of psychology.

[66]  Min-ju Lu,et al.  Effects of skill training on working memory capacity , 2007 .

[67]  P. Kidwell,et al.  The trouble with computers: Usefulness, usability and productivity , 1996, IEEE Annals of the History of Computing.

[68]  D S Emmerich,et al.  An investigation of the facilitation of simple auditory reaction time by predictable background stimuli , 1989, Perception & psychophysics.

[69]  David Patterson The trouble with multi-core , 2010, IEEE Spectrum.

[70]  H. M. Shih,et al.  The history of human factors and ergonomics , 2000 .

[71]  Baptist Liefooghe,et al.  Working memory costs of task switching. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[72]  Allen Newell,et al.  The psychology of human-computer interaction , 1983 .

[73]  Erik Brynjolfsson,et al.  The productivity paradox of information technology , 1993, CACM.

[74]  Marc Hassenzahl,et al.  User experience - a research agenda , 2006, Behav. Inf. Technol..

[75]  U. Foehr MEDIA MULTITASKING AMONG AMERICAN YOUTH: PREVALENCE, PREDICTORS AND PAIRINGS , 2006 .

[76]  Niels Taatgen,et al.  Toward a unified theory of the multitasking continuum: from concurrent performance to task switching, interruption, and resumption , 2009, CHI.

[77]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[78]  R. Larsen,et al.  Comparison of Inhibition in Two Timed Reaction Tasks: The Color and Emotion Stroop Tasks , 2008, The Journal of psychology.

[79]  E. T. Klemmer Time uncertainty in simple reaction time. , 1956, Journal of experimental psychology.

[80]  N. Meiran,et al.  Component Processes in Task Switching , 2000, Cognitive Psychology.

[81]  Harvey A. Taub,et al.  A comparison of young adult and old groups on various digit span tasks. , 1972 .

[82]  Thomas K. Landauer,et al.  Trouble with Computers: Usefulness, Usability, and Productivity , 1996 .

[83]  Dieter Fuhrmann User Centered Design An Integrated Approach , 2016 .

[84]  Richard T. Walls,et al.  ADULT LEARNERS: NEW NORMS ON THE NELSON-DENNY READING TEST FOR HEALTHCARE PROFESSIONALS , 2002 .

[85]  T. Salthouse The processing-speed theory of adult age differences in cognition. , 1996, Psychological review.

[86]  Marie P. Panepinto Voluntary versus Forced Task Switching , 2010 .

[87]  Ben Shneiderman,et al.  The Craft of Information Visualization: Readings and Reflections , 2003 .

[88]  Eric Brill,et al.  Improving web search ranking by incorporating user behavior information , 2006, SIGIR.

[89]  Chandler Stolp,et al.  The Visual Display of Quantitative Information , 1983 .