Matrix function optimization under weighted boundary constraints and its applications in network control.

The matrix function optimization under weighted boundary constraints on the matrix variables is investigated in this work. An "index-notation-arrangement based chain rule" (I-Chain rule) is introduced to obtain the gradient of a matrix function. By doing this, we propose the weighted trace-constraint-based projected gradient method (WTPGM) and weighted orthornormal-constraint-based projected gradient method (WOPGM) to locate a point of minimum of an objective/cost function of matrix variables iteratively subject to weighted trace constraint and weighted orthonormal constraint, respectively. New techniques are implemented to establish the convergence property of both algorithms. In addition, compared with the existing scheme termed "orthornormal-constraint-based projected gradient method" (OPGM) that requires the gradient has to be represented by the multiplication of a symmetrical matrix and the matrix variable itself, such a condition has been relaxed in WOPGM. Simulation results show the effectiveness of our methods not only in network control but also in other learning problems. We believe that the results reveal interesting physical insights in the field of network control and allow extensive applications of matrix function optimization problems in science and engineering.

[1]  Dit-Yan Yeung,et al.  A Convex Formulation of Modularity Maximization for Community Detection , 2011, IJCAI.

[2]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[3]  Charles W. Wampler,et al.  Manipulator Inverse Kinematic Solutions Based on Vector Formulations and Damped Least-Squares Methods , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[4]  Henk A. L. Kiers,et al.  Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems , 2002, Comput. Stat. Data Anal..

[5]  Yee Lian Chew,et al.  Network control principles predict neuron function in the Caenorhabditis elegans connectome , 2017, Nature.

[6]  P. Lancaster,et al.  The theory of matrices : with applications , 1985 .

[7]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[8]  Ziyang Meng,et al.  Behaviors of networks with antagonistic interactions and switching topologies , 2016, Autom..

[9]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[10]  Jack Levine,et al.  Curvature Collineations: A Fundamental Symmetry Property of the Space‐Times of General Relativity Defined by the Vanishing Lie Derivative of the Riemann Curvature Tensor , 1969 .

[11]  Hong Cheng,et al.  Trace Norm Regularized CANDECOMP/PARAFAC Decomposition With Missing Data , 2015, IEEE Transactions on Cybernetics.

[12]  Wei Yu,et al.  Iterative water-filling for Gaussian vector multiple-access channels , 2001, IEEE Transactions on Information Theory.

[13]  G. Arfken.  Mathematical Methods for Physicists , 1967 .

[14]  Y.-Y. Liu,et al.  The fundamental advantages of temporal networks , 2016, Science.

[15]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[16]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[17]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[18]  J. Perdew,et al.  Density-Functional Theory for Fractional Particle Number: Derivative Discontinuities of the Energy , 1982 .

[19]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[20]  Jon C. Dattorro,et al.  Convex Optimization & Euclidean Distance Geometry , 2004 .

[21]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[22]  Fuad E. Alsaadi,et al.  An Integrated Approach to Global Synchronization and State Estimation for Nonlinear Singularly Perturbed Complex Networks , 2015, IEEE Transactions on Cybernetics.

[23]  Hongyu Zhao,et al.  Normalized modularity optimization method for community identification with degree adjustment. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  T. R. Hughes,et al.  Mathematical foundations of elasticity , 1982 .

[25]  Ziyang Meng,et al.  Boundary Constraints for Minimum Cost Control of Directed Networks , 2017, IEEE Transactions on Cybernetics.

[26]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[27]  Shuicheng Yan,et al.  Graph embedding: a general framework for dimensionality reduction , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  R. Aris.  Vectors, Tensors and the Basic Equations of Fluid Mechanics , 1962 .

[29]  Erik W. Grafarend,et al.  Fourth order Taylor–Kármán structured covariance tensor for gravity gradient predictions by means of the Hankel transformation , 2015 .

[30]  Jan A Snyman,et al.  Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms , 2005 .

[31]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[32]  D. Ward,et al.  Directional Derivative Calculus and Optimality Conditions in Nonsmooth Mathematical Programming , 1989 .

[33]  Avinash C. Kak,et al.  PCA versus LDA , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[35]  Min Han,et al.  A fully automatic ocular artifact removal from EEG based on fourth-order tensor method , 2014 .

[36]  Lei Deng,et al.  Enabling Controlling Complex Networks with Local Topological Information , 2018, Scientific Reports.

[37]  Jie Ren,et al.  Controlling complex networks: How much energy is needed? , 2012, Physical review letters.

[38]  Wei Dai,et al.  Convergence of Gradient Descent for Low-Rank Matrix Approximation , 2015, IEEE Transactions on Information Theory.

[39]  Guoqi Li,et al.  Minimum-cost control of complex networks , 2015 .

[40]  A. Bloch,et al.  A variational problem on Stiefel manifolds , 2006, math/0609038.

[41]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .

[42]  Jimmie Gilbert,et al.  Linear Algebra and Matrix Theory , 1991 .

[43]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[44]  Xiaochun Cao,et al.  A Unified Semi-Supervised Community Detection Framework Using Latent Space Graph Regularization , 2015, IEEE Transactions on Cybernetics.

[45]  J. A. Schouten.  Tensor analysis for physicists , 1955 .

[46]  Are Hjrungnes,et al.  Complex-Valued Matrix Derivatives: With Applications in Signal Processing and Communications , 2011 .

[47]  Stephen Lin,et al.  Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval , 2007, IEEE Transactions on Image Processing.

[48]  Masaaki Harada.  Extremal Type I Zk-Codes and k-Frames of Odd Unimodular Lattices , 2015, IEEE Trans. Inf. Theory.

[49]  Benjamin Pfaff,et al.  Handbook Of Image And Video Processing , 2016 .

[50]  Ziyang Meng,et al.  Optimization on matrix manifold based on gradient information and its applications in network control , 2018, Physica A: Statistical Mechanics and its Applications.

[51]  P. Gǎvruţa,et al.  On the duality of fusion frames , 2007 .

[52]  Karim M. Abadir,et al.  Matrix Algebra: Notation , 2005 .

[53]  Xuelong Li,et al.  A Unified Learning Framework for Single Image Super-Resolution , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[54]  Rick S. Blum,et al.  Optimized signaling for MIMO interference systems with feedback , 2003, IEEE Trans. Signal Process..

[55]  Katherine K. Perkins,et al.  Observations on student difficulties with mathematics in upper-division electricity and magnetism , 2012 .

[56]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[57]  Dong Xu,et al.  Trace Ratio vs. Ratio Trace for Dimensionality Reduction , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.