Robot-assisted surgery: an emerging platform for human neuroscience research

Classic studies in human sensorimotor control use simplified tasks to uncover fundamental control strategies employed by the nervous system. Such simple tasks are critical for isolating specific features of motor, sensory, or cognitive processes, and for inferring causality between these features and observed behavioral changes. However, it remains unclear how these theories translate to complex sensorimotor tasks or to natural behaviors. Part of the difficulty in performing such experiments has been the lack of appropriate tools for measuring complex motor skills in real-world contexts. Robot-assisted surgery (RAS) provides an opportunity to overcome these challenges by enabling unobtrusive measurements of user behavior. In addition, a continuum of tasks with varying complexity—from simple tasks such as those in classic studies to highly complex tasks such as a surgical procedure—can be studied using RAS platforms. Finally, RAS includes a diverse participant population of inexperienced users all the way to expert surgeons. In this perspective, we illustrate how the characteristics of RAS systems make them compelling platforms to extend many theories in human neuroscience, as well as, to develop new theories altogether.

[1]  J. Randall Flanagan,et al.  Coding and use of tactile signals from the fingertips in object manipulation tasks , 2009, Nature Reviews Neuroscience.

[2]  A. Moinzadeh,et al.  Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. , 2009, Urology.

[3]  M. Tresch,et al.  The case for and against muscle synergies , 2022 .

[4]  Blake Hannaford,et al.  Raven-II: An Open Platform for Surgical Robotics Research , 2013, IEEE Transactions on Biomedical Engineering.

[5]  Guang-Zhong Yang,et al.  Perceptual Docking for Robotic Control , 2008, MIAR.

[6]  Karen J. Reynolds,et al.  Virtual reality surgical simulator software development tools , 2013, J. Simulation.

[7]  Erlend Fagertun Hofstad,et al.  A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance , 2013, Surgical Endoscopy.

[8]  C Ghez,et al.  Learning of Visuomotor Transformations for Vectorial Planning of Reaching Trajectories , 2000, The Journal of Neuroscience.

[9]  Firas Mawase,et al.  Evidence for predictive control in lifting series of virtual objects , 2010, Experimental Brain Research.

[10]  A. Okamura Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[11]  Helman Stern,et al.  Most Probable Longest Common Subsequence for Recognition of Gesture Character Input , 2013, IEEE Transactions on Cybernetics.

[12]  G. Rognini,et al.  Extending the Body to Virtual Tools Using a Robotic Surgical Interface: Evidence from the Crossmodal Congruency Task , 2012, PloS one.

[13]  Ferdinando A. Mussa-Ivaldi,et al.  Perception of Delayed Stiffness , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[14]  Wolf-Fritz Riekert Extracting area objects from raster image data , 1993, IEEE Computer Graphics and Applications.

[15]  Lee W. White,et al.  Content and construct validation of a robotic surgery curriculum using an electromagnetic instrument tracker. , 2012, The Journal of urology.

[16]  Mark Evans Breaking IT down , 2012 .

[17]  Ilana Nisky,et al.  Learning and generalization in an isometric visuomotor task. , 2015, Journal of neurophysiology.

[18]  J Randall Flanagan,et al.  Separate Contributions of Kinematic and Kinetic Errors to Trajectory and Grip Force Adaptation When Transporting Novel Hand-Held Loads , 2013, The Journal of Neuroscience.

[19]  D. Wolpert,et al.  Principles of sensorimotor learning , 2011, Nature Reviews Neuroscience.

[20]  E Bizzi,et al.  Motor learning through the combination of primitives. , 2000, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[21]  Reza Shadmehr,et al.  A memory of errors in sensorimotor learning , 2014, Science.

[22]  R. Ivry,et al.  The coordination of movement: optimal feedback control and beyond , 2010, Trends in Cognitive Sciences.

[23]  Timothy M. Kowalewski,et al.  Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: a prospective randomized controlled study. , 2013, Journal of the American College of Surgeons.

[24]  D. Sternad,et al.  Decomposition of variability in the execution of goal-oriented tasks: three components of skill improvement. , 2004, Journal of experimental psychology. Human perception and performance.

[25]  David I. Lee,et al.  In vivo validation of a system for haptic feedback of tool vibrations in robotic surgery , 2013, Surgical Endoscopy.

[26]  P. Morasso Spatial control of arm movements , 2004, Experimental Brain Research.

[27]  T. Judkins,et al.  Objective evaluation of expert and novice performance during robotic surgical training tasks , 2009, Surgical Endoscopy.

[28]  Michael A Liss,et al.  Robotic Surgical Simulation , 2013, Cancer journal.

[29]  P. Dasgupta,et al.  Current status of validation for robotic surgery simulators – a systematic review , 2013, BJU international.

[30]  J. Randall Flanagan,et al.  Motor learning of novel dynamics is not represented in a single global coordinate system: evaluation of mixed coordinate representations and local learning , 2013, Journal of neurophysiology.

[31]  Norberto F. Ezquerra,et al.  Interactively deformable models for surgery simulation , 1993, IEEE Computer Graphics and Applications.

[32]  Paul Glazier Movement Variability in the Golf Swing , 2011, Research quarterly for exercise and sport.

[33]  D. Sternad,et al.  Variability in motor learning: relocating, channeling and reducing noise , 2009, Experimental Brain Research.

[34]  Masaru Ishii,et al.  Surgical Task and Skill Classification from Eye Tracking and Tool Motion in Minimally Invasive Surgery , 2010, MICCAI.

[35]  Yohsuke R. Miyamoto,et al.  Temporal structure of motor variability is dynamically regulated and predicts motor learning ability , 2014, Nature Neuroscience.

[36]  Dagmar Sternad,et al.  Extrinsic contributions to movement variability in human object manipulation , 2014, 2014 40th Annual Northeast Bioengineering Conference (NEBEC).

[37]  Konrad P Kording,et al.  In Praise of “False” Models and Rich Data , 2010, Journal of motor behavior.

[38]  Aldo A. Faisal,et al.  The Manipulative Complexity of Lower Paleolithic Stone Toolmaking , 2010, PloS one.

[39]  R Shadmehr,et al.  Spatial Generalization from Learning Dynamics of Reaching Movements , 2000, The Journal of Neuroscience.

[40]  J. Klein,et al.  Breaking It Down Is Better: Haptic Decomposition of Complex Movements Aids in Robot-Assisted Motor Learning , 2012, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[41]  E Bizzi,et al.  Motor learning by field approximation. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[42]  T. Flash,et al.  The coordination of arm movements: an experimentally confirmed mathematical model , 1985, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[43]  Hiroshi Imamizu,et al.  Human cerebellar activity reflecting an acquired internal model of a new tool , 2000, Nature.

[44]  K. J. Kuchenbecker,et al.  Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery , 2015, Surgical Endoscopy.

[45]  Allison M. Okamura,et al.  Modeling of Tool-Tissue Interactions for Computer-Based Surgical Simulation: A Literature Review , 2008, PRESENCE: Teleoperators and Virtual Environments.

[46]  K. A. Ericsson,et al.  Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. , 2004, Academic medicine : journal of the Association of American Medical Colleges.

[47]  Bernhard Weber,et al.  The Effects of Force Feedback on Surgical Task Performance: A Meta-Analytical Integration , 2014, EuroHaptics.

[48]  Ilana Nisky,et al.  Perception of Stiffness with Force Feedback Delay , 2014, Multisensory Softness.

[49]  Blandine Bril,et al.  Coordination strategies used in stone knapping. , 2013, American journal of physical anthropology.

[50]  Wolfgang Taube,et al.  In Experts, underlying processes that drive visuomotor adaptation are different than in Novices , 2015, Front. Hum. Neurosci..

[51]  M. Kawato,et al.  Modular organization of internal models of tools in the human cerebellum , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[52]  A. Meraney,et al.  da Vinci Skills Simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance. , 2012, Urology.

[53]  J. Patton,et al.  Can Robots Help the Learning of Skilled Actions? , 2009, Exercise and sport sciences reviews.

[54]  Ferdinando A. Mussa-Ivaldi,et al.  A Regression and Boundary-Crossing-Based Model for the Perception of Delayed Stiffness , 2008, IEEE Transactions on Haptics.

[55]  F A Mussa-Ivaldi,et al.  Adaptive representation of dynamics during learning of a motor task , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[56]  J. Lackner,et al.  Motor control and learning in altered dynamic environments , 2005, Current Opinion in Neurobiology.

[57]  L Nazarko Breaking down. , 1992, Nursing the elderly : in hospital, homes and the community.

[58]  K. T. Ramesh,et al.  Modelling of non-linear elastic tissues for surgical simulation , 2010, Computer methods in biomechanics and biomedical engineering.

[59]  Paola Cesari,et al.  Body-goal Variability Mapping in an Aiming Task , 2006, Biological Cybernetics.

[60]  Paolo Dario,et al.  Modelling and Evaluation of Surgical Performance Using Hidden Markov Models , 2006, IEEE Transactions on Biomedical Engineering.

[61]  J. Kaouk,et al.  Fundamental skills of robotic surgery: a multi-institutional randomized controlled trial for validation of a simulation-based curriculum. , 2013, Urology.

[62]  Giulio Rognini,et al.  Force feedback facilitates multisensory integration during robotic tool use , 2013, Experimental Brain Research.

[63]  Henry C. Lin,et al.  JHU-ISI Gesture and Skill Assessment Working Set ( JIGSAWS ) : A Surgical Activity Dataset for Human Motion Modeling , 2014 .

[64]  Kris S Chaisanguanthum,et al.  Motor Variability Arises from a Slow Random Walk in Neural State , 2014, The Journal of Neuroscience.

[65]  Konrad Kording,et al.  The Database for Reaching Experiments and Models , 2013, PloS one.

[66]  Michael I. Jordan,et al.  Constrained and unconstrained movements involve different control strategies. , 1997, Journal of neurophysiology.

[67]  R. Shadmehr,et al.  Biological Learning and Control: How the Brain Builds Representations, Predicts Events, and Makes Decisions , 2012 .

[68]  Blake Hannaford,et al.  Generalized approach for modeling minimally invasive surgery as a stochastic process using a discrete Markov model , 2006, IEEE Transactions on Biomedical Engineering.

[69]  J. Krakauer,et al.  Human sensorimotor learning: adaptation, skill, and beyond , 2011, Current Opinion in Neurobiology.

[70]  Allison M. Okamura,et al.  Uncontrolled Manifold Analysis of Arm Joint Angle Variability During Robotic Teleoperation and Freehand Movement of Surgeons and Novices , 2014, IEEE Transactions on Biomedical Engineering.

[71]  Jonathan B Dingwell,et al.  Trial-to-trial dynamics and learning in a generalized, redundant reaching task. , 2013, Journal of neurophysiology.

[72]  S. Schaal The Computational Neurobiology of Reaching and Pointing — A Foundation for Motor Learning by Reza Shadmehr and Steven P. Wise , 2007 .

[73]  Zhiwei Luo,et al.  On the dynamic version of the minimum hand jerk criterion , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[74]  Pheng-Ann Heng,et al.  A virtual training system for maxillofacial surgery using advanced haptic feedback and immersive workbench , 2014, The international journal of medical robotics + computer assisted surgery : MRCAS.

[75]  Vipul Patel,et al.  Fundamentals of robotic surgery: a course of basic robotic surgery skills based upon a 14‐society consensus template of outcomes measures and curriculum development , 2014, The international journal of medical robotics + computer assisted surgery : MRCAS.

[76]  Claude Ghez,et al.  Separate adaptive mechanisms for controlling trajectory and final position in reaching. , 2007, Journal of neurophysiology.

[77]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.

[78]  Raz Leib,et al.  Minimum acceleration with constraints of center of mass: a unified model for arm movements and object manipulation. , 2012, Journal of neurophysiology.

[79]  E. Bizzi,et al.  Linear combinations of primitives in vertebrate motor control. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[80]  I. Gill,et al.  Face, content and construct validity of a novel robotic surgery simulator. , 2011, The Journal of urology.

[81]  Tamar Flash,et al.  Motor primitives in vertebrates and invertebrates , 2005, Current Opinion in Neurobiology.

[82]  Katherine J. Kuchenbecker,et al.  Tool Contact Acceleration Feedback for Telerobotic Surgery , 2011, IEEE Transactions on Haptics.

[83]  Reza Shadmehr,et al.  Optimizing effort: increased efficiency of motor memory with time away from practice. , 2015, Journal of neurophysiology.

[84]  Michael I. Jordan,et al.  Optimal feedback control as a theory of motor coordination , 2002, Nature Neuroscience.

[85]  Gregor Schöner,et al.  Use of the uncontrolled manifold (UCM) approach to understand motor variability, motor equivalence, and self-motion. , 2014, Advances in experimental medicine and biology.

[86]  Yasmin L. Hashambhoy,et al.  Neural Correlates of Reach Errors , 2005, The Journal of Neuroscience.

[87]  Allison M. Okamura,et al.  Grip Force Control during Virtual Object Interaction: Effect of Force Feedback, Accuracy Demands, and Training , 2014, IEEE Transactions on Haptics.

[88]  Gregor Schöner,et al.  Toward a new theory of motor synergies. , 2007, Motor control.

[89]  Christopher D Mah,et al.  Manipulating objects with internal degrees of freedom: evidence for model-based control. , 2002, Journal of neurophysiology.

[90]  Guang-Zhong Yang,et al.  Gaze-Contingent Motor Channelling and Haptic Constraints for Minimally Invasive Robotic Surgery , 2008, MICCAI.

[91]  Nigel W. John,et al.  The Role of Haptics in Medical Training Simulators: A Survey of the State of the Art , 2011, IEEE Transactions on Haptics.

[92]  Henry C. Lin,et al.  Towards automatic skill evaluation: Detection and segmentation of robot-assisted surgical motions , 2006, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[93]  Maurice A. Smith,et al.  Motor Memory Is Encoded as a Gain-Field Combination of Intrinsic and Extrinsic Action Representations , 2012, Journal of Neuroscience.

[94]  Ulman Lindenberger,et al.  Coordination of degrees of freedom and stabilization of task variables in a complex motor skill: expertise-related differences in cello bowing , 2013, Experimental Brain Research.

[95]  Myriam Curet,et al.  Construct validity of nine new inanimate exercises for robotic surgeon training using a standardized setup , 2013, Surgical Endoscopy.

[96]  F. A. Mussa-Ivaldi,et al.  Does the motor control system use multiple models and context switching to cope with a variable environment? , 2002, Experimental Brain Research.

[97]  Karol Miller,et al.  Meshless algorithm for soft tissue cutting in surgical simulation , 2014, Computer methods in biomechanics and biomedical engineering.

[98]  A. Okamura,et al.  Effects of robotic manipulators on movements of novices and surgeons , 2014, Surgical Endoscopy.

[99]  Elspeth M McDougall,et al.  Validation of surgical simulators. , 2007, Journal of endourology.

[100]  Mark L. Latash,et al.  The role of kinematic redundancy in adaptation of reaching , 2006, Experimental Brain Research.

[101]  Tamar Flash,et al.  Models of human movement: Trajectory planning and inverse kinematics studies , 2013, Robotics Auton. Syst..