Structural health monitoring for bolt loosening via a non-invasive vibro-haptics human–machine cooperative interface

For the last two decades, developments in damage detection algorithms have greatly increased the potential for autonomous decisions about structural health. However, we are still struggling to build autonomous tools that can match the ability of a human to detect and localize the quantity of damage in structures. Therefore, there is a growing interest in merging the computational and cognitive concepts to improve the solution of structural health monitoring (SHM). The main object of this research is to apply the human–machine cooperative approach on a tower structure to detect damage. The cooperation approach includes haptic tools to create an appropriate collaboration between SHM sensor networks, statistical compression techniques and humans. Damage simulation in the structure is conducted by releasing some of the bolt loads. Accelerometers are bonded to various locations of the tower members to acquire the dynamic response of the structure. The obtained accelerometer results are encoded in three different ways to represent them as a haptic stimulus for the human subjects. Then, the participants are subjected to each of these stimuli to detect the bolt loosened damage in the tower. Results obtained from the human–machine cooperation demonstrate that the human subjects were able to recognize the damage with an accuracy of 88 ± 20.21% and response time of 5.87 ± 2.33 s. As a result, it is concluded that the currently developed human–machine cooperation SHM may provide a useful framework to interact with abstract entities such as data from a sensor network.

[1]  Alessandra Angelucci,et al.  Induction of visual orientation modules in auditory cortex , 2000, Nature.

[2]  F. Chang,et al.  Bio-inspired stretchable network-based intelligent composites , 2013 .

[3]  L. Weiskrantz.  Blindsight : a case study and implications , 1986 .

[4]  J M Hoc,et al.  From human – machine interaction to human – machine cooperation , 2000, Ergonomics.

[5]  Charles R. Farrar,et al.  Machine learning algorithms for damage detection under operational and environmental variability , 2011 .

[6]  Charles R. Farrar,et al.  Structural Health Monitoring: A Machine Learning Perspective , 2012 .

[7]  M. Merzenich,et al.  The sense of flutter-vibration evoked by stimulation of the hairy skin of primates: Comparison of human sensory capacity with the responses of mechanoreceptive afferents innervating the hairy skin of monkeys , 2004, Experimental Brain Research.

[8]  Kenneth J. Loh,et al.  Recent Advances in Skin-Inspired Sensors Enabled by Nanotechnology , 2012 .

[9]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[10]  H B Barlow,et al.  PATTERN RECOGNITION AND THE RESPONSES OF SENSORY NEURONS * , 1969, Annals of the New York Academy of Sciences.

[11]  A. Fraioli,et al.  Sensation magnitude of vibrotactile stimuli , 1969 .

[12]  Johannes Stallkamp,et al.  Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition , 2012, Neural Networks.

[13]  Christian Cipriani,et al.  A Miniature Vibrotactile Sensory Substitution Device for Multifingered Hand Prosthetics , 2012, IEEE Transactions on Biomedical Engineering.

[14]  M. Rowe,et al.  Vibrotactile frequency discrimination in human hairy skin. , 2006, Journal of neurophysiology.

[15]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[16]  Mark J. Schulz,et al.  Development of novel single-wall carbon nanotube–epoxy composite ply actuators , 2005 .

[17]  Serge Debernard,et al.  Human-machine cooperation: Toward an activity regulation assistance for different air traffic control levels , 1994, Int. J. Hum. Comput. Interact..

[18]  David Mascareñas,et al.  A Vibro-Haptic Human-Machine Interface for Structural Health Monitoring Applications , 2013 .

[19]  David D. L. Mascarenas,et al.  Development of a novel human-machine interface exploiting sensor substitution for structural health monitoring , 2013, 2013 IEEE RO-MAN.

[20]  Karon E. MacLean,et al.  The Role of Choice in Longitudinal Recall of Meaningful Tactile Signals , 2008, 2008 Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems.

[21]  Ronald T. Verrillo.  Psychophysics of vibrotactile stimulation , 1983 .

[22]  Yon Visell,et al.  Tactile sensory substitution: Models for enaction in HCI , 2009, Interact. Comput..

[23]  N. Kotov,et al.  Tailoring Piezoresistive Sensitivity of Multilayer Carbon Nanotube Composite Strain Sensors , 2008 .

[24]  M. Sur,et al.  Visual behaviour mediated by retinal projections directed to the auditory pathway , 2000, Nature.

[25]  Charles R. Farrar,et al.  A vibro-haptic human–machine interface for structural health monitoring , 2014 .

[26]  G.S. Dhillon,et al.  Direct neural sensory feedback and control of a prosthetic arm , 2005, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[27]  Allen Cheung,et al.  The application of statistical pattern recognition methods for damage detection to field data , 2008 .

[28]  A. Kargov,et al.  Design and Evaluation of a Low-Cost Force Feedback System for Myoelectric Prosthetic Hands , 2006 .

[29]  M. Paradiso,et al.  Neuroscience: Exploring the Brain , 1996 .

[30]  J. B. Pittenger,et al.  Human Echolocation as a Basic Form of Perception and Action , 1995 .

[31]  Karl Schulte,et al.  Load and health monitoring in glass fibre reinforced composites with an electrically conductive nanocomposite epoxy matrix , 2008 .

[32]  Jernej Tonejc,et al.  Pattern Recognition in Collective Cognitive Systems: Hybrid Human-Machine Learning (HHML) By Heterogeneous Ensembles , 2010, IC-AI.

[33]  Kenneth J. Loh,et al.  In situ strain monitoring of fiber-reinforced polymers using embedded piezoresistive nanocomposites , 2010 .

[34]  Yang Wang,et al.  A mobile sensing system for structural health monitoring: design and validation , 2010, Smart Materials and Structures.

[35]  Kenneth J. Loh,et al.  Bio-inspired Sensors for Structural Health Monitoring , 2015 .

[36]  Jessica Block,et al.  Comparing Statistical Classification with a Vibro-Tactile Human–Machine Interface for Structural Health Monitoring , 2014 .

[37]  Suong V. Hoa,et al.  Failure detection and monitoring in polymer matrix composites subjected to static and dynamic loads using carbon nanotube networks , 2009 .

[38]  M. Rowe,et al.  Perceived pitch of vibrotactile stimuli: effects of vibration amplitude, and implications for vibration frequency coding. , 1990, The Journal of physiology.

[39]  Hoon Sohn,et al.  Damage diagnosis using time series analysis of vibration signals , 2001 .