Efficient Neural Network Verification via Layer-based Semidefinite Relaxations and Linear Cuts

We introduce an efficient and tight layer-based semidefinite relaxation for verifying local robustness of neural networks. The improved tightness is the result of the combination between semidefinite relaxations and linear cuts. We obtain a computationally efficient method by decomposing the semidefinite formulation into layerwise constraints. By leveraging on chordal graph decompositions, we show that the formulation here presented is provably tighter than current approaches. Experiments on a set of benchmark networks show that the approach here proposed enables the verification of more instances compared to other relaxation methods. The results also demonstrate that the SDP relaxation here proposed is one order of magnitude faster than previous SDP methods.

[1]  Alessio Lomuscio,et al.  An approach to reachability analysis for feed-forward ReLU neural networks , 2017, ArXiv.

[2]  Yang Zheng,et al.  Chordal decomposition in operator-splitting methods for sparse semidefinite programs , 2017, Mathematical Programming.

[3]  Matthew Mirman,et al.  Fast and Effective Robustness Certification , 2018, NeurIPS.

[4]  Ian Goodfellow,et al.  Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming , 2020, NeurIPS.

[5]  Richard Y. Zhang.  On the Tightness of Semidefinite Relaxations for Certifying Robustness to Adversarial Examples , 2020, NeurIPS.

[6]  Christian Tjandraatmadja,et al.  Strong mixed-integer programming formulations for trained neural networks , 2018, Mathematical Programming.

[7]  Pushmeet Kohli,et al.  A Dual Approach to Scalable Verification of Deep Networks , 2018, UAI.

[8]  Yang Zheng,et al.  Chordal sparsity in control and optimization of large-scale systems , 2019 .

[9]  Rüdiger Ehlers,et al.  Formal Verification of Piece-Wise Linear Feed-Forward Neural Networks , 2017, ATVA.

[10]  Manfred Morari,et al.  Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programming , 2019, ArXiv.

[11]  Junfeng Yang,et al.  Efficient Formal Safety Analysis of Neural Networks , 2018, NeurIPS.

[12]  Martin S. Andersen,et al.  Chordal Graphs and Semidefinite Optimization , 2015, Found. Trends Optim..

[13]  Martin Vechev,et al.  Beyond the Single Neuron Convex Barrier for Neural Network Certification , 2019, NeurIPS.

[14]  Kurt M. Anstreicher,et al.  Institute for Mathematical Physics Semidefinite Programming versus the Reformulation–linearization Technique for Nonconvex Quadratically Constrained Quadratic Programming Semidefinite Programming versus the Reformulation-linearization Technique for Nonconvex Quadratically Constrained , 2022 .

[15]  J. Zico Kolter,et al.  Provable defenses against adversarial examples via the convex outer adversarial polytope , 2017, ICML.

[16]  Juan Pablo Vielma,et al.  The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verification , 2020, NeurIPS.

[17]  Kazuo Murota,et al.  Exploiting Sparsity in Semidefinite Programming via Matrix Completion I: General Framework , 2000, SIAM J. Optim..

[18]  Yoshio Okamoto,et al.  B-453 User's Manual for SparseCoLO: Conversion Methods for SPARSE COnic-form Linear Optimization Problems , 2009 .

[19]  Cho-Jui Hsieh,et al.  A Convex Relaxation Barrier to Tight Robustness Verification of Neural Networks , 2019, NeurIPS.

[20]  Inderjit S. Dhillon,et al.  Towards Fast Computation of Certified Robustness for ReLU Networks , 2018, ICML.

[21]  Mykel J. Kochenderfer,et al.  The Marabou Framework for Verification and Analysis of Deep Neural Networks , 2019, CAV.

[22]  J. Lofberg,et al.  YALMIP : a toolbox for modeling and optimization in MATLAB , 2004, 2004 IEEE International Conference on Robotics and Automation (IEEE Cat. No.04CH37508).

[23]  Russ Tedrake,et al.  Evaluating Robustness of Neural Networks with Mixed Integer Programming , 2017, ICLR.

[24]  Mykel J. Kochenderfer,et al.  Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks , 2017, CAV.

[25]  Aditi Raghunathan,et al.  Semidefinite relaxations for certifying robustness to adversarial examples , 2018, NeurIPS.

[26]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[27]  P. Henriksen,et al.  Efficient Neural Network Verification via Adaptive Refinement and Adversarial Search , 2020, ECAI.

[28]  Bo Li,et al.  SoK: Certified Robustness for Deep Neural Networks , 2020, ArXiv.

[29]  Timon Gehr,et al.  An abstract domain for certifying neural networks , 2019, Proc. ACM Program. Lang..