Neuromechanics of a Button Press

To press a button, a finger must push down and pull up with the right force and timing. How the motor system succeeds in button-pressing, in spite of neural noise and lacking direct access to the mechanism of the button, is poorly understood. This paper investigates a unifying account based on neuromechanics. Mechanics is used to model muscles controlling the finger that contacts the button. Neurocognitive principles are used to model how the motor system learns appropriate muscle activations over repeated strokes though relying on degraded sensory feedback. Neuromechanical simulations yield a rich set of predictions for kinematics, dynamics, and user performance and may aid in understanding and improving input devices. We present a computational implementation and evaluate predictions for common button types.

[1]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[2]  Richard A Satterlie,et al.  Neuromechanics: an integrative approach for understanding motor control. , 2007, Integrative and comparative biology.

[3]  Jukka Corander,et al.  Inferring Cognitive Models from Data using Approximate Bayesian Computation , 2016, CHI.

[4]  Philip S. E. Farrell,et al.  Perceptual control and layered protocols in interface design: I. Fundamental concepts , 1999, Int. J. Hum. Comput. Stud..

[5]  W. Ashby,et al.  An Introduction to Cybernetics , 1957 .

[6]  Hannes Eisler,et al.  Psychophysical issues in the study of time perception. , 2008 .

[7]  H. Gomi Implicit online corrections of reaching movements , 2008, Current Opinion in Neurobiology.

[8]  Dan Odell,et al.  The typing performance and preference costs of reducing tactile feedback and tactile landmarks in tablet keyboards , 2014 .

[9]  Viktor Mikhaĭlovich Glushkov,et al.  An Introduction to Cybernetics , 1957, The Mathematical Gazette.

[10]  G. Aschersleben Temporal Control of Movements in Sensorimotor Synchronization , 2002, Brain and Cognition.

[11]  G Aschersleben,et al.  Synchronizing actions with events: The role of sensory information , 1995, Perception & psychophysics.

[12]  J. Morton,et al.  Perceptual centers (P-centers). , 1976 .

[13]  Vincent Hayward,et al.  Mechanical behavior of the fingertip in the range of frequencies and displacements relevant to touch. , 2012, Journal of biomechanics.

[14]  Hans Brunner,et al.  Effects of Keyboard Design and Typing Skill on User Keyboard Preferences and Throughput Performance , 1984 .

[15]  Stephen A. Brewster,et al.  Investigating the effectiveness of tactile feedback for mobile touchscreens , 2008, CHI.

[16]  R. Radwin,et al.  Computer key switch force-displacement characteristics and short-term effects on localized fatigue. , 1999, Ergonomics.

[17]  Stephen A. Brewster,et al.  The effects of walking speed on target acquisition on a touchscreen interface , 2011, Mobile HCI.

[18]  Michael J. Hove,et al.  Spatiotemporal relations and movement trajectories in visuomotor synchronization , 2010 .

[19]  Joanna Bergstrom-Lehtovirta,et al.  A simple index for multimodal flexibility , 2010, CHI.

[20]  W. Smutz,et al.  A structural fingertip model for simulating of the biomechanics of tactile sensation. , 2004, Medical engineering & physics.

[21]  J. Hohwy The Predictive Mind , 2013 .

[22]  Antti Oulasvirta,et al.  Impact Activation Improves Rapid Button Pressing , 2018, CHI.

[23]  Demetri Terzopoulos,et al.  Heads up!: biomechanical modeling and neuromuscular control of the neck , 2006, ACM Trans. Graph..

[24]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[25]  Antti Oulasvirta,et al.  Moving Target Selection: A Cue Integration Model , 2018, CHI.

[26]  Robert G. Radwin,et al.  Activation Force and Travel effects on Overexertion in Repetitive Key Tapping , 1997, Hum. Factors.

[27]  Jörg Müller,et al.  Control Theoretic Models of Pointing , 2017, ACM Trans. Comput. Hum. Interact..

[28]  José Gaspar,et al.  Haptics of in-car radio buttons and its relationship with engineering parameters , 2017 .

[29]  Andruid Kerne,et al.  ZeroTouch: an optical multi-touch and free-air interaction architecture , 2012, CHI.

[30]  A. Landi Human Hand Function , 2007 .

[31]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[32]  Eve E. Hoggan,et al.  Boxer: a multimodal collision technique for virtual objects , 2017, ICMI.

[33]  W. Mansell,et al.  A biopsychosocial model based on negative feedback and control , 2014, Front. Hum. Neurosci..

[34]  Reinhard Blickhan,et al.  Nonlinearities make a difference: comparison of two common Hill-type models with real muscle , 2008, Biological Cybernetics.

[35]  Lena H Ting,et al.  Neuromechanics of muscle synergies for posture and movement , 2007, Current Opinion in Neurobiology.

[36]  R G Dong,et al.  Simulation of mechanical responses of fingertip to dynamic loading. , 2002, Medical engineering & physics.

[37]  Antti Oulasvirta,et al.  Informing the Design of Novel Input Methods with Muscle Coactivation Clustering , 2015, ACM Trans. Comput. Hum. Interact..

[38]  Antti Oulasvirta,et al.  Modelling Error Rates in Temporal Pointing , 2016, CHI.

[39]  S. Schaal,et al.  Computational motor control in humans and robots , 2005, Current Opinion in Neurobiology.

[40]  Nicolas Roussel,et al.  Characterizing Latency in Touch and Button-Equipped Interactive Systems , 2017, UIST.

[41]  Mary E Sesto,et al.  Performance and touch characteristics of disabled and non-disabled participants during a reciprocal tapping task using touch screen technology. , 2012, Applied ergonomics.

[42]  M. Treisman Temporal discrimination and the indifference interval. Implications for a model of the "internal clock". , 1963, Psychological monographs.

[43]  Marc O. Ernst,et al.  A Bayesian view on multimodal cue integration , 2006 .

[44]  Mark L. Nagurka,et al.  Measurement of Stiffness and Damping Characteristics of Computer Keyboard Keys , 2005 .

[45]  M. M. Taylor,et al.  Perceptual control and layered protocols in interface design: II. The general protocol grammar , 1999, Int. J. Hum. Comput. Stud..

[46]  O. Schmitt The heat of shortening and the dynamic constants of muscle , 2017 .

[47]  Antti Oulasvirta,et al.  Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry , 2015, CHI.

[48]  John M. Flach,et al.  Control Theory for Humans: Quantitative Approaches To Modeling Performance , 2002 .

[49]  Gisa Aschersleben,et al.  A psychophysical approach to action timing , 2004 .

[50]  B J Martin,et al.  The effect of keyboard keyswitch make force on applied force and finger flexor muscle activity. , 1997, Ergonomics.

[51]  James R. Lewis,et al.  Keys and Keyboards , 1997 .

[52]  Hiroyuki Kajimoto,et al.  Vibration Feedback Latency Affects Material Perception During Rod Tapping Interactions , 2017, IEEE Transactions on Haptics.

[53]  R. Enoka Neuromechanics of Human Movement , 2001 .

[54]  Kenichi Akagi A Computer Keyboard Key Feel Study in Performance and Preference , 1992 .

[55]  Steven C. Seow Information Theoretic Models of HCI: A Comparison of the Hick-Hyman Law and Fitts' Law , 2005, Hum. Comput. Interact..

[56]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[57]  David Harris,et al.  Perceptual control and feedback control in the analysis of complex tasks , 2014 .

[58]  Etienne Burdet,et al.  Human Robotics: Neuromechanics and Motor Control , 2013 .

[59]  Peter W Johnson,et al.  Differences in typing forces, muscle activity, comfort, and typing performance among virtual, notebook, and desktop keyboards. , 2014, Applied ergonomics.

[60]  Kathleen M. Potosnak Chapter 21 – Keys and Keyboards , 1988 .

[61]  W. T. Powers Behavior, the control of perception , 1973 .

[62]  Eftychios Sifakis,et al.  Comprehensive biomechanical modeling and simulation of the upper body , 2009, TOGS.

[63]  C. Palmer,et al.  Synchronization of Timing and Motion 435 , 2022 .

[64]  Anil K. Seth,et al.  The cybernetic Bayesian brain: from interoceptive inference to sensorimotor contingencies , 2014 .

[65]  Andruid Kerne,et al.  intangibleCanvas: free-air finger painting on a projected canvas , 2011, CHI Extended Abstracts.

[66]  Stephen A. Brewster,et al.  Towards the Temporally Perfect Virtual Button: Touch-Feedback Simultaneity and Perceived Quality in Mobile Touchscreen Press Interactions , 2014, TAP.

[67]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[68]  B. Repp Sensorimotor synchronization: A review of the tapping literature , 2005, Psychonomic bulletin & review.

[69]  Ramesh Balasubramaniam,et al.  Two different processes for sensorimotor synchronization in continuous and discontinuous rhythmic movements , 2009, Experimental Brain Research.

[70]  Y. Yarom,et al.  Resonance, oscillation and the intrinsic frequency preferences of neurons , 2000, Trends in Neurosciences.

[71]  Macy Ann Valk An Experiment to Study Touchscreen “Button” Design , 1985 .

[72]  Jeongmin Son,et al.  TapBoard: making a touch screen keyboard more touchable , 2013, CHI Extended Abstracts.

[73]  Arun P Sripati,et al.  Predicting the timing of spikes evoked by tactile stimulation of the hand. , 2010, Journal of neurophysiology.

[74]  Konrad Paul Kording,et al.  Review TRENDS in Cognitive Sciences Vol.10 No.7 July 2006 Special Issue: Probabilistic models of cognition Bayesian decision theory in sensorimotor control , 2022 .

[75]  Gordon D Logan,et al.  Warning: This keyboard will deconstruct— The role of the keyboard in skilled typewriting , 2010, Psychonomic bulletin & review.

[76]  E. Schröger,et al.  Psychophysics beyond sensation : laws and invariants of human cognition , 2004 .