Optimal Feature Manipulation Attacks Against Linear Regression

In this paper, we investigate how to manipulate the coefficients obtained via linear regression by adding carefully designed poisoning data points to the dataset or modify the original data points. Given the energy budget, we first provide the closed-form solution of the optimal poisoning data point when our target is modifying one designated regression coefficient. We then extend the analysis to the more challenging scenario where the attacker aims to change one particular regression coefficient while making others to be changed as small as possible. For this scenario, we introduce a semidefinite relaxation method to design the best attack scheme. Finally, we study a more powerful adversary who can perform a rank-one modification on the feature matrix. We propose an alternating optimization method to find the optimal rank-one modification matrix. Numerical examples are provided to illustrate the analytical results obtained in this paper.

[1]  Jen-Tzung Chien,et al.  Recursive Bayesian Linear Regression for Adaptive Classification , 2009, IEEE Transactions on Signal Processing.

[2]  Bin Li,et al.  CNN-Based Adversarial Embedding for Image Steganography , 2019, IEEE Transactions on Information Forensics and Security.

[3]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[4]  Paul Barford,et al.  Data Poisoning Attacks against Autoregressive Models , 2016, AAAI.

[5]  Andrew L. Beam,et al.  Adversarial attacks on medical machine learning , 2019, Science.

[6]  Chang Liu,et al.  Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[7]  Martin J. Wainwright,et al.  Log-determinant relaxation for approximate inference in discrete Markov random fields , 2006, IEEE Transactions on Signal Processing.

[8]  Robert Sabourin,et al.  Characterizing and Evaluating Adversarial Examples for Offline Handwritten Signature Verification , 2019, IEEE Transactions on Information Forensics and Security.

[9]  Etienne Perot,et al.  Deep Reinforcement Learning framework for Autonomous Driving , 2017, Autonomous Vehicles and Machines.

[10]  J. Lasserre,et al.  Detecting global optimality and extracting solutions in GloptiPoly , 2003 .

[11]  Bhaskar D. Rao,et al.  Statistical analysis of subspace-based estimation of reduced-rank linear regressions , 2002, IEEE Trans. Signal Process..

[12]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[13]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[14]  Marc Teboulle,et al.  Finding a Global Optimal Solution for a Quadratically Constrained Fractional Quadratic Problem with Applications to the Regularized Total Least Squares , 2006, SIAM J. Matrix Anal. Appl..

[15]  Abdelhak M. Zoubir,et al.  Beamforming via Nonconvex Linear Regression , 2016, IEEE Transactions on Signal Processing.

[16]  R. Michael Buehrer,et al.  Evaluating Adversarial Evasion Attacks in the Context of Wireless Communications , 2019, IEEE Transactions on Information Forensics and Security.

[17]  Nikos D. Sidiropoulos,et al.  Fast Approximation Algorithms for a Class of Non-convex QCQP Problems Using First-Order Methods , 2017, IEEE Transactions on Signal Processing.

[18]  Qianmu Li,et al.  Adversarial Deep Ensemble: Evasion Attacks and Defenses for Malware Detection , 2020, IEEE Transactions on Information Forensics and Security.

[19]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[20]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[21]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[22]  Steve Juggins,et al.  Weighted averaging partial least squares regression (WA-PLS): an improved method for reconstructing environmental variables from species assemblages , 1993, Hydrobiologia.

[23]  Oguz Akbilgic,et al.  A novel Hybrid RBF Neural Networks model as a forecaster , 2013, Statistics and Computing.

[24]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[25]  Siddharth Garg,et al.  BadNets: Evaluating Backdooring Attacks on Deep Neural Networks , 2019, IEEE Access.

[26]  Pan He,et al.  Adversarial Examples: Attacks and Defenses for Deep Learning , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Samy Bengio,et al.  Adversarial Machine Learning at Scale , 2016, ICLR.

[28]  Sergios Theodoridis,et al.  Robust Linear Regression Analysis— A Greedy Approach , 2014, IEEE Transactions on Signal Processing.

[29]  Blaine Nelson,et al.  Support Vector Machines Under Adversarial Label Noise , 2011, ACML.

[30]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[31]  Patrick D. McDaniel,et al.  Making machine learning robust against adversarial inputs , 2018, Commun. ACM.

[32]  M. Laurent.  Sums of Squares, Moment Matrices and Optimization Over Polynomials , 2009 .

[33]  Jean B. Lasserre,et al.  Sparse-BSOS: a bounded degree SOS hierarchy for large scale polynomial optimization with sparsity , 2016, Mathematical Programming Computation.

[34]  Xin Yan,et al.  Linear Regression Analysis: Theory and Computing , 2009 .

[35]  Dawn Xiaodong Song,et al.  Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning , 2017, ArXiv.

[36]  Amir Beck,et al.  On Minimizing Quadratically Constrained Ratio of Two Quadratic Functions ∗ , 2010 .

[37]  Constantine Caramanis,et al.  Robust Matrix Completion and Corrupted Columns , 2011, ICML.

[38]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[39]  Fabio Roli,et al.  Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning , 2018, CCS.

[40]  N. Shephard,et al.  Econometric Analysis of Realized Covariation: High Frequency Based Covariance, Regression, and Correlation in Financial Economics , 2004 .

[41]  Shuguang Cui,et al.  On the Adversarial Robustness of Subspace Learning , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Marc Teboulle,et al.  A convex optimization approach for minimizing the ratio of indefinite quadratic functions over an ellipsoid , 2009, Math. Program..

[43]  Hyunsoo Yoon,et al.  Selective Audio Adversarial Example in Evasion Attack on Speech Recognition System , 2020, IEEE Transactions on Information Forensics and Security.

[44]  Daniel L. Pimentel-Alarcón,et al.  Adversarial principal component analysis , 2017, 2017 IEEE International Symposium on Information Theory (ISIT).

[45]  Bevan K. Youse,et al.  Introduction to real analysis , 1972 .

[46]  Xiaojin Zhu,et al.  Using Machine Teaching to Identify Optimal Training-Set Attacks on Machine Learners , 2015, AAAI.

[47]  Blaine Nelson,et al.  Poisoning Attacks against Support Vector Machines , 2012, ICML.