Development of a haptic-based rey-osterrieth complex figure testing and training system with computer scoring and force feedback rehabilitation functions

This research develops a new system for assessment and rehabilitation of motor skills in Traumatic Brain Injury (TBI) patients. By using contemporary computing technologies, including virtual reality (VR), haptic devices and telecommunications, the system can work as an alternative to traditional labor intensive and expensive diagnosis and rehabilitation procedures for TBI patients. This study also introduces a general approach to the design and prototyping of a haptic-based VR System for motor skill assessment and rehabilitation. The Rey-Osterrieth Complex Figure (ROCF) is a neuropsychological test that has been used to manually assess various cognitive operations, such as perceptual apprehension, attention and control, graphomotor coordination, etc. The focus of this research was on developing a VR-based ROCF test system incorporating a haptic interface with functions to support testing protocols, based on specific user requirements, and to facilitate automated scoring of tests and quantitative test result output. Computer-based tests have advantages over traditional paper-based tests for motor skill assessment including automated recording of complete behavior information in the drawing process, simplifying the job of examiners, and reducing biases in evaluations. Haptic technology was used in the simulator as both an input and force feedback device. Haptic devices have the capability of recording user performance in six degrees of freedom (6 DOF) in the drawing process and providing force feedback in 3 DOF. This means they have the potential to provide user performance data on the traditional three dimensional (3D) method of ROCF testing. Also users may experience a more realistic feeling of drawing when using the haptic device versus using a tablet or tablet PC. To develop the ROCF scoring system, advanced technologies of handwriting/pattern recognition were reviewed and adapted. A software application was created for recording of freehand drawings, recognition of strokes and normalization of strokes for unit scoring. ROCF scoring and analysis reports can be generated automatically with the system. The haptic-based VR system was extended in capability for motor skill rehabilitation. Numerical models were used to describe motor skill assessment results and to parameterize the rehabilitation training application. The prototype system demonstrates the potential for using advanced information and VR technology to build more effective and intelligent tools for healthcare. This study identified the major challenges in developing a fully automated ROCF test system, including hardware options and criteria for choosing the proper devices for human performance. The software challenges are also identified and algorithms were developed for use in the system. The prototype system is expected to change the process of motor-skill evaluation in clinical practice.

[1]  Theodosios Pavlidis,et al.  A Shape Analysis Model with Applications to a Character Recognition System , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Blake Hannaford,et al.  Stable haptic interaction with virtual environments , 1999, IEEE Trans. Robotics Autom..

[3]  Rachael M Collie,et al.  Rape: psychopathology, theory and treatment. , 2008, Clinical psychology review.

[4]  E. DeMaria,et al.  Training the novice in laparoscopy , 2002, Surgical Endoscopy And Other Interventional Techniques.

[5]  Heidi Sveistrup,et al.  Motor rehabilitation using virtual reality , 2004, Journal of NeuroEngineering and Rehabilitation.

[6]  Jean W. Wilkins,et al.  Organizational quality on the Rey-Osterrieth and Taylor Complex Figure Tests: A new scoring system. , 1993 .

[7]  R. Satava,et al.  Virtual Reality Training Improves Operating Room Performance: Results of a Randomized, Double-Blinded Study , 2002, Annals of surgery.

[8]  Maureen K. Holden,et al.  Virtual Environments for Motor Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[9]  J. Edward Colgate,et al.  Factors affecting the Z-Width of a haptic display , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[10]  King-Sun Fu,et al.  Syntactic Pattern Recognition And Applications , 1968 .

[11]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[12]  Antoine Geissbühler,et al.  The Digital Pen and Paper Technology: Implementation and Use in an Existing Clinical Information System , 2005, MIE.

[13]  Thenkurussi Kesavadas,et al.  Motor skill training assistance using haptic attributes , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[14]  Yuan-Shin Lee,et al.  Heterogeneous material modelling and virtual prototyping with 5-DOF haptic force feedback for product development , 2008 .

[15]  Jinhai Cai,et al.  Handwriting Recognition , 2003 .

[16]  Etienne Burdet,et al.  A robotic teacher of Chinese handwriting , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[17]  John Kenneth Salisbury,et al.  Haptic Rendering: Introductory Concepts , 2004, IEEE Computer Graphics and Applications.

[18]  P. Osterrieth.  Le test de copie d'une figure complexe , 1944 .

[19]  J. Edward Colgate,et al.  Passive robots and haptic displays based on nonholonomic elements , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[20]  Metin Akay,et al.  Virtual Reality for Health Care , 2001 .

[21]  M. Selby,et al.  Neuropsychological functioning in drug abusers. , 1998, Drug and alcohol dependence.

[22]  Luc Soler,et al.  Advances in virtual reality are wide ranging , 2002, BMJ : British Medical Journal.

[23]  Hans P. Moravec.  Obstacle avoidance and navigation in the real world by a seeing robot rover , 1980 .

[24]  M. Merzenich,et al.  Use-dependent alterations of movement representations in primary motor cortex of adult squirrel monkeys , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  Yuan-Shin Lee,et al.  Dexel-based force-torque rendering and volume updating for 5-DOF haptic product prototyping and virtual sculpting , 2004, Comput. Ind..

[26]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[27]  A. Rizzo,et al.  The application of virtual reality technology in rehabilitation. , 2001 .

[28]  Mark R. Cutkosky,et al.  Stable User-Specific Haptic Rendering of the Virtual Wall , 1996, Dynamic Systems and Control.

[29]  J. Bennett-Levy.  Determinants of performance on the Rey-Osterrieth Complex Figure Test: an analysis, and a new technique for single-case assessment. , 1984, The British journal of clinical psychology.

[30]  G. Riva,et al.  Virtual Environments in Clinical Psychology and Neuroscience: Methods and Techniques in Advanced Patient-Therapist Interaction , 1998 .

[31]  I Grant,et al.  Alcoholics' initial organizational and problem-solving skills predict learning and memory performance on the Rey–Osterrieth Complex Figure , 2000, Journal of the International Neuropsychological Society.

[32]  Rama Chellappa,et al.  Evaluation of pattern classifiers for fingerprint and OCR applications , 1994, Pattern Recognit..

[33]  A. Rey.  L'examen psychologique dans les cas d'encéphalopathie traumatique. (Les problems.). , 1941 .

[34]  J. Moline,et al.  Virtual reality for health care: a survey. , 1997, Studies in health technology and informatics.

[35]  David B. Kaber,et al.  The Utility of a Virtual Reality Locomotion Interface for Studying Gait Behavior , 2007, Hum. Factors.

[36]  Yuan-Shin Lee,et al.  Five-axis pencil-cut planning and virtual prototyping with 5-DOF haptic interface , 2004, Comput. Aided Des..

[37]  Jinhai Cai,et al.  Handwriting Recognition - Soft Computing and Probabilistic Approaches , 2003, Studies in Fuzziness and Soft Computing.

[38]  William Harry Maddox,et al.  Incremental Static Semantic Analysis , 1998 .

[39]  Marc M. Sebrechts,et al.  HANDBOOK OF VIRTUAL ENVIRONMENTS , 2014 .

[40]  A. Rey.  Lexamen psychologique : Dans les cas d'encephalopathie traumatique (Les problemes) , 1941 .

[41]  Jack Sklansky,et al.  Finding circles by an array of accumulators , 1975, Commun. ACM.

[42]  Albert A. Rizzo,et al.  Virtual Reality in Brain Damage Rehabilitation: Review , 2005, Cyberpsychology Behav. Soc. Netw..

[43]  M. Sano,et al.  Parameter identification of a fuzzy model on the handwriting process of an arabic letter , 2001, Proceedings 10th IEEE International Workshop on Robot and Human Interactive Communication. ROMAN 2001 (Cat. No.01TH8591).

[44]  Robert Stone,et al.  Virtual reality laparoscopic simulator for assessment in gynaecology , 2003, BJOG : an international journal of obstetrics and gynaecology.

[45]  E. Todorov,et al.  Virtual Environment Training Improves Motor Performance in Two Patients with Stroke: Case Report , 1999 .

[46]  Z Mubrin,et al.  Effects of occupational exposure to organic solvents upon cognitive performance. , 1990, Neurotoxicology and teratology.

[47]  L. Tupler,et al.  Reliability of the Rey-Osterrieth Complex Figure in use with memory-impaired patients. , 1995, Journal of clinical and experimental neuropsychology.

[48]  R McCloy,et al.  Virtual reality in surgery , 2001, BMJ : British Medical Journal.

[49]  Kay M. Stanney,et al.  Virtual environments , 2002 .

[50]  J. Strackee,et al.  A handwriting simulator. , 1962, Physics in medicine and biology.

[51]  RAOUF F. H. FARAG,et al.  Word-Level Recognition of Cursive Script , 1979, IEEE Transactions on Computers.

[52]  P. Satz,et al.  Comparison of four methods of scoring the Rey-Osterrieth complex figure drawing test on four age groups of normal elderly , 1992 .

[53]  Zoran Duric,et al.  Poster: Teaching letter writing using a programmable haptic device interface for children with handwriting difficulties , 2009, 2009 IEEE Symposium on 3D User Interfaces.

[54]  John C. Norcross,et al.  The face of 2010: A Delphi poll on the future of psychotherapy. , 2002 .

[55]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[56]  Edith Kaplan,et al.  The handbook of Rey-Osterrieth Complex Figure Usage : clinical and research applications , 2003 .

[57]  David Blatner,et al.  Silicon Mirage: The Art and Science of Virtual Reality , 1992 .

[58]  A. Kurz,et al.  Validation of the Addenbrooke’s Cognitive Examination for Detecting Early Alzheimer’s Disease and Mild Vascular Dementia in a German Population , 2006, Dementia and Geriatric Cognitive Disorders.

[59]  L. Cermak,et al.  Role of Perceptual and Organizational Factors in Amnesics' Recall of the Rey-Osterrieth Complex Figure: A Comparison of Three Amnesic Groups , 2000, Journal of clinical and experimental neuropsychology.

[60]  G. Riva,et al.  Communications Through Virtual Technologies: Identity, Community and Technology in the Communication Age , 2001 .

[61]  Stuart Russell,et al.  Statistical Visual Language Models for Ink Parsing , 2002 .

[62]  Richard Buse,et al.  A structural and relational approach to handwritten word recognition , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[63]  C Despont-Gros,et al.  The Digital Pen and Paper , 2005, Methods of Information in Medicine.

[64]  I. S. I. Abuhaiba,et al.  A fuzzy graph theoretic approach to recognize the totally unconstrained handwritten numerals , 1993, Pattern Recognit..

[65]  C Chinnock,et al.  Virtual reality in surgery and medicine. , 1994, Hospital technology series.

[66]  Paul G. Lesnik,et al.  Evidence for cerebellar-frontal subsystem changes in children treated with intrathecal chemotherapy for leukemia: enhanced data analysis using an effect size model. , 1998, Archives of neurology.

[67]  Gregory B. Newby,et al.  Virtual reality: Scientific and technological challenges , 1996 .

[68]  Alex Mihailidis,et al.  A haptic-robotic platform for upper-limb reaching stroke therapy: Preliminary design and evaluation results , 2007, Journal of NeuroEngineering and Rehabilitation.

[69]  Mark Billinghurst.  Review of Virtual reality technology (2nd Edition) by Grigore Burdea and Philippe Coiffet, Wiley-Interscience , 2003 .

[70]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[71]  Kenneth J. Ottenbacher,et al.  Virtual Reality in the Assessment of Selected Cognitive Function After Brain Injury , 2001, American journal of physical medicine & rehabilitation.

[72]  Y.K. Kim,et al.  Hand-writing Rehabilitation in the Haptic Virtual Environment , 2006, 2006 IEEE International Workshop on Haptic Audio Visual Environments and their Applications (HAVE 2006).

[73]  Maureen K. Holden,et al.  Use of Virtual Environments in Motor Learning and Rehabilitation , 2002 .

[74]  Philippe Coiffet,et al.  Virtual Reality Technology , 2003, Presence: Teleoperators & Virtual Environments.

[75]  Grigore C. Burdea,et al.  The Rutgers Master II-new design force-feedback glove , 2002 .

[76]  G Riva,et al.  Applications of Virtual Environments in Medicine , 2003, Methods of Information in Medicine.

[77]  Albert A. Rizzo,et al.  Virtual environment applications in clinical neuropsychology , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[78]  Saeid Nahavandi,et al.  Haptic handwriting aid for training and rehabilitation , 2005, 2005 IEEE International Conference on Systems, Man and Cybernetics.

[79]  Donglin Zeng,et al.  A comparative study of mobile electronic data entry systems for clinical trials data collection , 2006, Int. J. Medical Informatics.

[80]  G. R. Brown,et al.  A screening test for subtle cognitive impairment early in the course of HIV infection. , 1993, Psychosomatics.

[81]  W. Sturm,et al.  Neuropsychological assessment , 2007, Journal of Neurology.

[82]  Alan Liu,et al.  A Survey of Surgical Simulation: Applications, Technology, and Education , 2003, Presence: Teleoperators & Virtual Environments.