Accuracy-Robustness Trade-Off for Positively Weighted Neural Networks

This work proposes a new learning strategy for training a feedforward neural network subject to spectral norm and nonnegativity constraints. Our primary goal is to control the Lipschitz constant of the network in order to make it robust against adversarial perturbations of its inputs. We propose a stochastic projected gradient descent algorithm which allows us to adjust this constant in the training process. The algorithm is evaluated in the context of designing a fully connected network for Automatic Gesture Recognition based on EMG signals. We perform a comparison with the same architecture trained either in a standard manner or with simpler constraints. The obtained results highlight that a good accuracy-robustness trade-off can be achieved.

[1]  Clément Gosselin,et al.  Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning , 2018, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Jacek M. Zurada,et al.  Learning Understandable Neural Networks With Nonnegative Weight Constraints , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Elisabeth André,et al.  EMG-based hand gesture recognition for realtime biosignal interfacing , 2008, IUI '08.

[4]  Samy Bengio,et al.  Adversarial Machine Learning at Scale , 2016, ICLR.

[5]  Yuichi Yoshida,et al.  Spectral Normalization for Generative Adversarial Networks , 2018, ICLR.

[6]  Moustapha Cissé,et al.  Parseval Networks: Improving Robustness to Adversarial Examples , 2017, ICML.

[7]  Rami N. Khushaba,et al.  Electromyogram (EMG) feature reduction using Mutual Components Analysis for multifunction prosthetic fingers control , 2012, 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV).

[8]  Corneliu Burileanu,et al.  Automatic EMG-based Hand Gesture Recognition System using Time-Domain Descriptors and Fully-Connected Neural Networks , 2019, 2019 42nd International Conference on Telecommunications and Signal Processing (TSP).

[9]  Émilie Chouzenoux,et al.  Dual Block-Coordinate Forward-Backward Algorithm with Application to Deconvolution and Deinterlacing of Video Sequences , 2017, Journal of Mathematical Imaging and Vision.

[10]  Manfred Morari,et al.  Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networks , 2019, NeurIPS.

[11]  Manfredo Atzori,et al.  Electromyography data for non-invasive naturally-controlled robotic hand prostheses , 2014, Scientific Data.

[12]  Joan Bruna,et al.  Intriguing properties of neural networks , 2013, ICLR.

[13]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[14]  Cem Anil,et al.  Sorting out Lipschitz function approximation , 2018, ICML.

[15]  Andrés Felipe Ruiz Olaya,et al.  Deep neural network for EMG signal classification of wrist position: Preliminary results , 2017, 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI).

[16]  Patrick L. Combettes,et al.  Deep Neural Network Structures Solving Variational Inequalities , 2018, Set-Valued and Variational Analysis.

[17]  P. L. Combettes,et al.  Lipschitz Certificates for Neural Network Structures Driven by Averaged Activation Operators , 2019 .

[18]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[19]  Patrick L. Combettes,et al.  Lipschitz Certificates for Layered Network Structures Driven by Averaged Activation Operators , 2019, SIAM J. Math. Data Sci..

[20]  P. L. Combettes,et al.  Dualization of Signal Recovery Problems , 2009, 0907.0436.

[21]  Micah Sherr,et al.  Hidden Voice Commands , 2016, USENIX Security Symposium.

[22]  Kevin Scaman,et al.  Lipschitz regularity of deep neural networks: analysis and efficient estimation , 2018, NeurIPS.