Understanding Visual Impairment: A CA-CV Approach for Cognitive Computer Vision

Partially sighted people face multiple challenges in their daily lives. For example, they are not confident about their interactions with other people, mainly because they are not sure of other people's expressions, feelings or intentions. In many cases, this situation results in social issues such as isolation and marginalisation among others. The field of computer vision has attempted to provide solutions to some of the problems people with visual impairment face. Currently, computer vision technologies are able to identify almost any element available in our environment. However, most of the recognition processes are trained based on image classification. The current proposal pushes the boundaries of computer vision (henceforth CV) powered by image categorisation by evolving it to the next stage, namely cognitive computer vision. We aim to develop a computer algorithm to enhance machine vision by incorporating a conversational analytic (henceforth CA) approach. This intelligence will be embedded into a hardware device with the objective of providing real-time feedback to visually impaired users via speech output, thus reducing the gap between technology and human-like augmented senses.

[1]  J. M. Atkinson.  Structures of Social Action: Contents , 1985 .

[2]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 2015 .

[3]  Alexandra Paxton,et al.  Frame-differencing methods for measuring bodily synchrony in conversation , 2012, Behavior Research Methods.

[4]  Ali Borji,et al.  Negative results in computer vision: A perspective , 2017, Image Vis. Comput..

[5]  Qiang Ji,et al.  Data-Free Prior Model for Facial Action Unit Recognition , 2013, IEEE Transactions on Affective Computing.

[6]  V. Balasubramanian,et al.  Using a haptic belt to convey non-verbal communication cues during social interactions to individuals who are blind , 2008, 2008 IEEE International Workshop on Haptic Audio visual Environments and Games.

[7]  Antoni Malet,et al.  Keplerian illusions: Geòmetrical pictures vs optical images in Kepler's visual theory , 1990 .

[8]  Maja Pantic,et al.  A Dynamic Appearance Descriptor Approach to Facial Actions Temporal Modeling , 2014, IEEE Transactions on Cybernetics.

[9]  Emanuel A. Schegloff,et al.  Practices and actions: Boundary cases of other‐initiated repair , 1997 .

[10]  Jennifer J. Hunter,et al.  Vision science and adaptive optics, the state of the field , 2017, Vision Research.

[11]  Larry S. Davis,et al.  Robust Human Detection under Occlusion by Integrating Face and Person Detectors , 2009, ICB.

[12]  Anthony J. Liddicoat,et al.  An Introduction to Conversation Analysis , 2007 .

[13]  H. Petrie,et al.  Exploring Intrinsic and Extrinsic Motivations to Participate in a Crowdsourcing Project to Support Blind and Partially Sighted Students , 2016, UD.

[14]  Paul J. Taylor,et al.  AMAB: Automated measurement and analysis of body motion , 2013, Behavior research methods.

[15]  William Robson Schwartz,et al.  Detecting humans under partial occlusion using Markov logic networks , 2010, PerMIS.

[16]  Monica Gori,et al.  Neuroscience and Biobehavioral Reviews , 2022 .

[17]  Charles C. Ragin,et al.  Constructing social research :the unity and diversity of method , 1996 .

[18]  Stephen J. Romano,et al.  Role Playing , 2008, Behavior modification.

[19]  Helen Petrie,et al.  User Requirements for Technologies for Personal Communication and Information Use for Deafblind People , 2002, ICCHP.

[20]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[21]  Helen Petrie,et al.  Comparing Concurrent and Retrospective Verbal Protocols for Blind and Sighted Users , 2015, INTERACT.

[22]  Helen Petrie,et al.  Types of Problems Elicited by Verbal Protocols for Blind and Sighted Participants , 2016, ICCHP.