A nonlinear contextually aware prompting system (N-CAPS) to assist workers with intellectual and developmental disabilities to perform factory assembly tasks: system overview and pilot testing

Abstract Purpose: This paper presents a new cognitive assistive technology, nonlinear contextually aware prompting system (N-CAPS) that uses advanced sensing and artificial intelligence to monitor and provide assistance to workers with cognitive disabilities during a factory assembly task. Methods: The N-CAPS system was designed through the application of various computer vision and artificial intelligence algorithms that allows the system to track a user during a specific assembly task, and then provide verbal and visual prompts to the worker as needed. A pilot study was completed with the N-CAPS solution in order to investigate whether it was an appropriate intervention. Four participants completed the required assembly task five different times, using the N-CAPS system. Results: The participants completed all of the trials that they attempted with 85.7% of the steps completed without assistance from the job coach. Of the 85.7% of steps completed independently, 32.5% of these were completed in response to prompts given by N-CAPS. Overall system accuracy was 83.3%, the overall sensitivity was 86.2% and the overall specificity was 82.4%. Conclusions: The results from the study were positive in that they showed that this type of technology does have merit with this population. Implications for Rehabilitation It provides a concise summary of the importance of work in the lives of people with intellectual disabilities and how technology can support this life goal. It describes the first artificially intelligent system designed to support workers with intellectually disabilities. It provides evidence that individuals with intellectual disabilities can perform a work task in response to technology.

[1]  R. P. Chakrabarty,et al.  SURVEY OF INCOME AND PROGRAM PARTICIPATION , 1990 .

[2]  Stefan Carmien,et al.  MAPS: creating socio-technical environments in support of distributed cognition for people with cognitive impairments and their caregivers , 2004, CHI EA '04.

[3]  S. Folstein,et al.  "Mini-mental state". A practical method for grading the cognitive state of patients for the clinician. , 1975, Journal of psychiatric research.

[4]  Stefan Carmien,et al.  End user programming and context responsiveness in handheld prompting systems for persons with cognitive disabilities and caregivers , 2005, CHI Extended Abstracts.

[5]  Linda C. Mechling,et al.  Computer-Based Video Instruction to Teach Young Adults with Moderate Intellectual Disabilities to Perform Multiple Step, Job Tasks in a Generalized Setting , 2007 .

[6]  Albert J. Pautler.  Workforce Education: Issues for the New Century , 1999 .

[7]  M. Bergman.  The Benefits of a Cognitive Orthotic in Brain Injury Rehabilitation , 2002, The Journal of head trauma rehabilitation.

[8]  C. Bodine,et al.  Technology for improving cognitive function. A workshop sponsored by the U.S. Interagency Committee on Disability Research (ICDR): Reports from working groups , 2006, Disability and rehabilitation.

[9]  Yu-Chi Tsai.  Kinempt: a Kinect-based prompting system to transition autonomously through vocational tasks for individuals with cognitive impairments , 2012, ASSETS '12.

[10]  Jesse Hoey,et al.  The use of an intelligent prompting system for people with dementia , 2007, Interactions.

[11]  W. Alexander.  American psychiatric association. , 2008, P & T : a peer-reviewed journal for formulary management.

[12]  David F. Cihak,et al.  Generalized use of a handheld prompting system. , 2007, Research in developmental disabilities.

[13]  H. Kraemer,et al.  A new brief instrument for assessing decisional capacity for clinical research. , 2007, Archives of general psychiatry.

[14]  F. Rusch.  Book Review: Real Work for Real Pay: Inclusive Employment for People with Disabilities , 2007 .

[15]  S. Dumont,et al.  Determinants of return to work among spinal cord injury patients: A literature review , 2007 .

[16]  J. Barbenel,et al.  The efficacy of an intelligent cognitive orthosis to facilitate handwashing by persons with moderate to severe dementia , 2004 .

[17]  H. Yanco,et al.  Prompting Devices : A Survey of Memory Aids for Task Sequencing , 2010 .

[18]  Steven E. Stock,et al.  Promoting Independent Performance of Transition-Related Tasks Using a Palmtop PC-Based Self-Directed Visual and Auditory Prompting System , 2005 .

[19]  Debra Schreckenghost,et al.  Distributed cognitive aid with scheduling and interactive task guidance. , 2008, Journal of rehabilitation research and development.

[20]  J. Schaller,et al.  Competitive Employment for People With Autism , 2005 .

[21]  J. Arango-Lasprilla,et al.  Community integration: current issues in cognitive and vocational rehabilitation for individuals with ABI. , 2009, Journal of rehabilitation research and development.

[22]  Mark F. O’Reilly,et al.  Comparing Video Prompting to Video Modeling for Teaching Daily Living Skills to Six Adults with Developmental Disabilities , 2006 .

[23]  M. Wolery,et al.  Comparison of response prompting procedures in teaching numeral identification to autistic subjects , 1988, Journal of autism and developmental disorders.

[24]  A. Mihailidis,et al.  The COACH prompting system to assist older adults with dementia through handwashing: An efficacy study , 2008, BMC geriatrics.

[25]  J. Lai,et al.  The impact of assistive technology and environmental interventions on function and living situation status with people who are ageing with developmental disabilities , 2002, Disability and rehabilitation.

[26]  Michael P. Brady,et al.  The Effects of Covert Audio Coaching on the Job Performance of Supported Employees , 2010 .

[27]  Kathleen Bodisch Lynch,et al.  Personal digital assistants as cognitive aids for individuals with severe traumatic brain injury: A community-based trial , 2008, Brain injury.

[28]  Alex Mihailidis,et al.  Quantitative analysis of formal caregivers' use of communication strategies while assisting individuals with moderate and severe Alzheimer's disease during oral care. , 2013, Journal of communication disorders.

[29]  Alex Zelinsky,et al.  Learning OpenCV---Computer Vision with the OpenCV Library (Bradski, G.R. et al.; 2008)[On the Shelf] , 2009, IEEE Robotics & Automation Magazine.

[30]  Yao-Jen Chang,et al.  Context-aware prompting to transition autonomously through vocational tasks for individuals with cognitive impairments , 2009, Assets '09.

[31]  M. Wehmeyer,et al.  Using Self-Monitoring to Increase Following-Direction Skills of Students with Moderate to Severe Disabilities in General Education , 2005 .

[32]  David Mank,et al.  Integrated employment or sheltered workshops: Preferences of adults with intellectual disabilities, their families, and staff , 2016 .

[33]  David F. Cihak,et al.  Use of a Handheld Prompting System to Transition Independently Through Vocational Tasks for Students with Moderate and Severe Intellectual Disabilities , 2008 .

[34]  Steven E. Stock,et al.  Enhancing Independent Task Performance for Individuals with Mental Retardation through Use of a Handheld Self-Directed Visual and Audio Prompting System. , 2002 .

[35]  G E Lancioni,et al.  Promoting Independent Task Performance by Persons with Severe Developmental Disabilities through a New Computer-Aided System , 2000, Behavior modification.

[36]  P. Wehman.  Employment for persons with disabilities: Where are we now and where do we need to go? , 2011 .

[37]  A. Mihailidis,et al.  Assistive technology for cognitive rehabilitation: State of the art , 2004 .

[38]  H. Kaye.  Stuck at the Bottom Rung: Occupational Characteristics of Workers with Disabilities , 2009, Journal of Occupational Rehabilitation.

[39]  Jesse Hoey,et al.  EMPOWERING ADULTS WITH A COGNITIVE DISABILITY THROUGH INCLUSION OF NON-LINEAR CONTEXT AWARE PROMPTING TECHNOLOGY ( N-CAPS ) , 2012 .

[40]  Assistive Technology and Environmental Intervention (AT-EI) Impact on the Activity and Life Roles of Aging Adults with Developmental Disabilities , 2000 .

[41]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[42]  Michael L. Wehmeyer,et al.  The State of the States in Developmental Disabilities (5th ed.) , 1999 .

[43]  Richard T. Connis.  The effects of sequential pictorial cues, self-recording, and praise on the job task sequencing of retarded adults. , 1979, Journal of applied behavior analysis.

[44]  Jesse W. Johnson,et al.  The Effectiveness of Using a Video iPod as a Prompting Device in Employment Settings , 2009 .

[45]  G E Lancioni,et al.  An overview of technological resources used in rehabilitation research with people with severe/profound and multiple disabilities. , 2001, Disability and rehabilitation.