Neural-inspired sensors enable sparse, efficient classification of spatiotemporal data

Significance Winged insects perform remarkable aerial feats in uncertain, complex fluid environments. This ability is enabled by sensation of mechanical forces to inform rapid corrections in body orientation. Curiously, mechanoreceptor neurons do not faithfully report forces; instead, they are activated by specific time histories of forcing. We find that, far from being a bug, neural encoding by biological sensors is a feature that acts as built-in temporal filtering superbly matched to detect body rotation. Indeed, this encoding further enables surprisingly efficient detection using only a small handful of neurons at key locations. Nature suggests smart data as an alternative strategy to big data, and neural-inspired sensors establish a paradigm in hyperefficient sensing of complex systems. Sparse sensor placement is a central challenge in the efficient characterization of complex systems when the cost of acquiring and processing data is high. Leading sparse sensing methods typically exploit either spatial or temporal correlations, but rarely both. This work introduces a sparse sensor optimization that is designed to leverage the rich spatiotemporal coherence exhibited by many systems. Our approach is inspired by the remarkable performance of flying insects, which use a few embedded strain-sensitive neurons to achieve rapid and robust flight control despite large gust disturbances. Specifically, we identify neural-inspired sensors at a few key locations on a flapping wing that are able to detect body rotation. This task is particularly challenging as the rotational twisting mode is three orders of magnitude smaller than the flapping modes. We show that nonlinear filtering in time, built to mimic strain-sensitive neurons, is essential to detect rotation, whereas instantaneous measurements fail. Optimized sparse sensor placement results in efficient classification with approximately 10 sensors, achieving the same accuracy and noise robustness as full measurements consisting of hundreds of sensors. Sparse sensing with neural-inspired encoding establishes an alternative paradigm in hyperefficient, embodied sensing of spatiotemporal data and sheds light on principles of biological sensing for agile flight control.

[1]  T. Daniel,et al.  Neural evidence supports a dual sensory-motor role for insect wings , 2017, Proceedings of the Royal Society B: Biological Sciences.

[2]  W S McCulloch,et al.  A logical calculus of the ideas immanent in nervous activity , 1990, The Philosophy of Artificial Intelligence.

[3]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[4]  Steven L. Brunton,et al.  Chaos as an intermittently forced linear system , 2016, Nature Communications.

[5]  Trevor J. Hastie,et al.  Sparse Discriminant Analysis , 2011, Technometrics.

[6]  Steven L. Brunton,et al.  Data-Driven Sparse Sensor Placement for Reconstruction: Demonstrating the Benefits of Exploiting Known Patterns , 2017, IEEE Control Systems.

[7]  C. Majidi Soft Robotics: A Perspective—Current Trends and Prospects for the Future , 2014 .

[8]  S. M. Walker,et al.  Smart wing rotation and trailing-edge vortices enable high frequency mosquito flight , 2017, Nature.

[9]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[10]  A. Cohen,et al.  Wake structures behind a swimming robotic lamprey with a passively flexible tail , 2012, Journal of Experimental Biology.

[11]  R. Hengstenberg,et al.  The halteres of the blowfly Calliphora , 1994, Journal of Comparative Physiology A.

[12]  J. DiCarlo,et al.  Using goal-driven deep learning models to understand sensory cortex , 2016, Nature Neuroscience.

[13]  Holger G Krapp,et al.  Nonlinear Integration of Visual and Haltere Inputs in Fly Neck Motor Neurons , 2009, The Journal of Neuroscience.

[14]  A. Smits,et al.  Energy harvesting eel , 2001 .

[15]  T. Daniel,et al.  The Journal of Experimental Biology 206, 2989-2997 © 2003 The Company of Biologists Ltd , 2003 .

[16]  T. Daniel,et al.  Shape, flapping and flexion: wing and fin design for forward flight. , 2001, The Journal of experimental biology.

[17]  Adrienne L. Fairhall,et al.  Analysis of Neuronal Spike Trains, Deconstructed , 2016, Neuron.

[18]  William Bialek,et al.  Analyzing Neural Responses to Natural Signals: Maximally Informative Dimensions , 2002, Neural Computation.

[19]  Steven L. Brunton,et al.  Sparse Sensor Placement Optimization for Classification , 2016, SIAM J. Appl. Math..

[20]  G. Nalbach The halteres of the blowfly Calliphora , 1993, Journal of Comparative Physiology A.

[21]  Daniel M. Vogt,et al.  Embedded 3D Printing of Strain Sensors within Highly Stretchable Elastomers , 2014, Advanced materials.

[22]  Steven L. Brunton,et al.  Exploiting sparsity and equation-free architectures in complex systems , 2014 .

[23]  S. Sane,et al.  Antennal Mechanosensors Mediate Flight Control in Moths , 2007, Science.

[24]  B. H. Dickerson,et al.  Control of moth flight posture is mediated by wing mechanosensory feedback , 2014, Journal of Experimental Biology.

[25]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[26]  M. Dickinson,et al.  Haltere-mediated equilibrium reflexes of the fruit fly, Drosophila melanogaster. , 1999, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[27]  Imraan A. Faruque,et al.  Dipteran insect flight dynamics. Part 1 Longitudinal motion about hover. , 2010, Journal of theoretical biology.

[28]  M. Dickinson,et al.  A comparison of visual and haltere-mediated equilibrium reflexes in the fruit fly Drosophila melanogaster , 2003, Journal of Experimental Biology.

[29]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[30]  D. Boyd,et al.  Six Provocations for Big Data , 2011 .

[31]  Steven L. Brunton,et al.  Data-Driven Sparse Sensor Placement , 2017, ArXiv.

[32]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[33]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[34]  H. Krapp,et al.  Sensory Systems and Flight Stability: What do Insects Measure and Why? , 2007 .

[35]  D. Lazer,et al.  The Parable of Google Flu: Traps in Big Data Analysis , 2014, Science.

[36]  M. Triantafyllou,et al.  An Efficient Swimming Machine , 1995 .

[37]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[38]  T. Daniel,et al.  A neural basis for gyroscopic force measurement in the halteres of Holorusia , 2008, Journal of Comparative Physiology A.

[39]  Kenneth Breuer,et al.  Aeromechanics of Membrane Wings with Implications for Animal Flight ArnoldSong, ∗ XiaodongTian, † EmilyIsraeli, ‡ RicardoGalvao, § KristinBishop, ¶ SharonSwartz, ∗∗ , 2008 .

[40]  John O Dabiri,et al.  Fish schooling as a basis for vertical axis wind turbine farm design , 2010, Bioinspiration & biomimetics.

[41]  B. R. Noack,et al.  Closed-Loop Turbulence Control: Progress and Challenges , 2015 .

[42]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[43]  Massimo Fornasier,et al.  Compressive Sensing , 2015, Handbook of Mathematical Methods in Imaging.

[44]  T. Collett,et al.  Visual control of flight behaviour in the hoverflySyritta pipiens L. , 1975, Journal of comparative physiology.

[45]  J. Sean Humbert,et al.  Robust Gust Rejection on a Micro-air Vehicle Using Bio-inspired Sensing , 2017 .

[46]  A. Fairhall,et al.  Encoding properties of haltere neurons enable motion feature detection in a biological gyroscope , 2010, Proceedings of the National Academy of Sciences.

[47]  A. Smits,et al.  Thrust production and wake structure of a batoid-inspired oscillating fin , 2005, Journal of Fluid Mechanics.

[48]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[49]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[50]  D. O’Carroll,et al.  Wide-field motion tuning in nocturnal hawkmoths , 2010, Proceedings of the Royal Society B: Biological Sciences.

[51]  B. H. Dickerson,et al.  A new twist on gyroscopic sensing: body rotations lead to torsion in flapping, flexing insect wings , 2015, Journal of The Royal Society Interface.

[53]  MajidiCarmel,et al.  Soft Robotics: A Perspective—Current Trends and Prospects for the Future , 2014 .

[54]  Steven L. Brunton,et al.  Koopman Invariant Subspaces and Finite Linear Representations of Nonlinear Dynamical Systems for Control , 2015, PloS one.

[55]  I. Faruque,et al.  Dipteran insect flight dynamics. Part 2: Lateral-directional motion about hover. , 2010, Journal of theoretical biology.

[56]  Jonathan P. Dyhr,et al.  Luminance-dependent visual processing enables moth flight in low light , 2015, Science.