Cyborg groups enhance face recognition in crowded environments

Recognizing a person in a crowded environment is a challenging, yet critical, visual-search task for both humans and machine-vision algorithms. This paper explores the possibility of combining a residual neural network (ResNet), brain-computer interfaces (BCIs) and human participants to create “cyborgs” that improve decision making. Human participants and a ResNet undertook the same face-recognition experiment. BCIs were used to decode the decision confidence of humans from their EEG signals. Different types of cyborg groups were created, including either only humans (with or without the BCI) or groups of humans and the ResNet. Cyborg groups decisions were obtained weighing individual decisions by confidence estimates. Results show that groups of cyborgs are significantly more accurate (up to 35%) than the ResNet, the average participant, and equally-sized groups of humans not assisted by technology. These results suggest that melding humans, BCI, and machine-vision technology could significantly improve decision-making in realistic scenarios.

[1]  M. Shadlen,et al.  Choice Certainty Is Informed by Both Evidence and Decision Time , 2014, Neuron.

[2]  Raúl Rojas,et al.  Semi-autonomous Car Control Using Brain Computer Interfaces , 2012, IAS.

[3]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[4]  David J. Robertson,et al.  Unfamiliar face recognition : Security, surveillance and smartphones , 2016 .

[5]  IEEE conference on computer vision and pattern recognition , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[6]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[7]  Narendra Ahuja,et al.  Detecting Faces in Images: A Survey , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  José del R. Millán,et al.  The role of shared-control in BCI-based telepresence , 2010, 2010 IEEE International Conference on Systems, Man and Cybernetics.

[9]  Stefan M. Herzog,et al.  Boosting medical diagnostics by pooling independent judgments , 2016, Proceedings of the National Academy of Sciences.

[10]  D. Helbing,et al.  How social influence can undermine the wisdom of crowd effect , 2011, Proceedings of the National Academy of Sciences.

[11]  T. Demiralp,et al.  What if you are not sure? Electroencephalographic correlates of subjective confidence level about a decision , 2012, Clinical Neurophysiology.

[12]  Caterina Cinel,et al.  Cross-modal illusory conjunctions between vision and touch. , 2002, Journal of experimental psychology. Human perception and performance.

[13]  Riccardo Poli,et al.  Collaborative Brain-Computer Interface for Aiding Decision-Making , 2014, PloS one.

[14]  G. Vanacker,et al.  Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[15]  Febo Cincotti,et al.  Tools for Brain-Computer Interaction: A General Concept for a Hybrid BCI , 2011, Front. Neuroinform..

[16]  Daniel J. Simons,et al.  Inattentional blindness , 2007, Scholarpedia.

[17]  Roger Ratcliff,et al.  Modeling confidence judgments, response times, and multiple choices in decision making: recognition memory and motion discrimination. , 2013, Psychological review.

[18]  Rob Jenkins,et al.  Face Recognition by Metropolitan Police Super-Recognisers , 2016, PloS one.

[19]  Yongkang Wong,et al.  Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition , 2011, CVPR 2011 WORKSHOPS.

[20]  Andres Laan,et al.  Rescuing Collective Wisdom when the Average Group Opinion Is Wrong , 2017, Front. Robot. AI.

[21]  Kait Clark,et al.  Visual search performance is predicted by both prestimulus and poststimulus electrical brain activity , 2016, Scientific Reports.

[22]  J M Hoc,et al.  From human – machine interaction to human – machine cooperation , 2000, Ergonomics.

[23]  Miguel P Eckstein,et al.  Visual search: a retrospective. , 2011, Journal of vision.

[24]  David Dunning,et al.  Overconfidence Among Beginners: Is a Little Learning a Dangerous Thing? , 2018, Journal of personality and social psychology.

[25]  Bethany L. Ojalehto,et al.  Systems of (non-)diversity , 2017, Nature Human Behaviour.

[26]  Jacob Goldberger,et al.  Distilling the wisdom of crowds: weighted aggregation of decisions on multiple issues , 2009, Autonomous Agents and Multi-Agent Systems.

[27]  S. Luck,et al.  Electrophysiological correlates of feature analysis during visual search. , 1994, Psychophysiology.

[28]  Paul M Bays,et al.  Temporal dynamics of encoding, storage, and reallocation of visual working memory. , 2011, Journal of vision.

[29]  Pawan Sinha,et al.  Face Recognition by Humans: Nineteen Results All Computer Vision Researchers Should Know About , 2006, Proceedings of the IEEE.

[30]  P. Latham,et al.  Confidence matching in group decision-making , 2017, Nature Human Behaviour.

[31]  Swami Sankaranarayanan,et al.  Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms , 2018, Proceedings of the National Academy of Sciences.

[32]  Craig K. Abbey,et al.  Neural decoding of collective wisdom with multi-brain computing , 2012, NeuroImage.

[33]  J. Baron,et al.  The Power of Social Influence on Estimation Accuracy , 2015 .

[34]  N. Kerr,et al.  Group performance and decision making. , 2004, Annual review of psychology.

[35]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[36]  Zhi-Hua Zhou,et al.  Face recognition from a single image per person: A survey , 2006, Pattern Recognit..

[37]  Riccardo Poli,et al.  Enhancement of Group Perception via a Collaborative Brain–Computer Interface , 2017, IEEE Transactions on Biomedical Engineering.

[38]  Nick Yeung,et al.  Shared Neural Markers of Decision Confidence and Error Detection , 2015, The Journal of Neuroscience.

[39]  Alexandre Pouget,et al.  Confidence and certainty: distinct probabilistic quantities for different goals , 2016, Nature Neuroscience.

[40]  V. Bruce,et al.  Face Recognition in Poor-Quality Video: Evidence From Security Surveillance , 1999 .

[41]  John A. Pyles,et al.  Dynamic Encoding of Face Information in the Human Fusiform Gyrus , 2014, Nature Communications.

[42]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[43]  Dan Bang,et al.  Making better decisions in groups , 2017, Royal Society Open Science.

[44]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[45]  Lauren E. Welbourne,et al.  Humans, but Not Deep Neural Networks, Often Miss Giant Targets in Scenes , 2017, Current Biology.

[46]  Gavin Brown,et al.  Individual Confidence-Weighting and Group Decision-Making. , 2017, Trends in ecology & evolution.

[47]  Riccardo Poli,et al.  Augmenting group performance in target-face recognition via collaborative brain-computer interfaces for surveillance applications , 2017, 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER).

[48]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[49]  Bahador Bahrami,et al.  Aggregated knowledge from a small number of debates outperforms the wisdom of large crowds , 2017, Nature Human Behaviour.

[50]  A. Burton,et al.  Crowd Effects in Unfamiliar Face Matching , 2013 .

[51]  Denis Fize,et al.  Speed of processing in the human visual system , 1996, Nature.

[52]  Leonard Branson,et al.  WHEN TWO HEADS ARE WORSE THAN ONE: IMPACT OF GROUP STYLE AND INFORMATION TYPE ON PERFORMANCE EVALUATION , 2010 .

[53]  Å. Austin,et al.  A cross-scale trophic cascade from large predatory fish to algae in coastal ecosystems , 2017, Proceedings of the Royal Society B: Biological Sciences.

[54]  A. Hunt,et al.  Human visual search behaviour is far from ideal , 2017, Proceedings of the Royal Society B: Biological Sciences.

[55]  Bahador Bahrami,et al.  Post-decisional accounts of biases in confidence , 2016, Current Opinion in Behavioral Sciences.

[56]  Riccardo Poli,et al.  Group Augmentation in Realistic Visual-Search Decisions via a Hybrid Brain-Computer Interface , 2017, Scientific Reports.

[57]  Rob Jenkins,et al.  Face detection dissociates from face identification , 2017 .