Matrix Function Optimization Problems Under Orthonormal Constraint

We investigate the matrix function optimization under the orthonormal constraint on the matrix variable. By introducing an index-notation-arrangement-based chain rule (I-Chain rule), we obtain the gradient of the cost function and propose a revisited orthonormal-constraint-based projected gradient method to locate a minimum of an objective/cost function of matrix variables iteratively subject to orthonormal constraint. To guarantee the convergence the proposed method, existing schemes require the gradient can be represented by the multiplication of a symmetrical matrix and the matrix variable itself. This condition has been relaxed in this paper. New techniques are proposed to establish the convergence property of the iterative algorithm. Simulation results show the effectiveness of our framework. This paper allows more extensive applications of matrix function optimization problems in science and engineering.

[1]  Bernhard Schölkopf,et al.  On a Kernel-Based Method for Pattern Recognition, Regression, Approximation, and Operator Inversion , 1998, Algorithmica.

[2]  P. Lancaster,et al.  The theory of matrices : with applications , 1985 .

[3]  Christos Faloutsos,et al.  Graph evolution: Densification and shrinking diameters , 2006, TKDD.

[4]  Jack Levine,et al.  Curvature Collineations: A Fundamental Symmetry Property of the Space‐Times of General Relativity Defined by the Vanishing Lie Derivative of the Riemann Curvature Tensor , 1969 .

[5]  R. Burt.  Social Contagion and Innovation: Cohesion versus Structural Equivalence , 1987, American Journal of Sociology.

[6]  P. Gǎvruţa,et al.  On the duality of fusion frames , 2007 .

[7]  Ziyang Meng,et al.  Boundary Constraints for Minimum Cost Control of Directed Networks , 2017, IEEE Transactions on Cybernetics.

[8]  J. A. Schouten.  Tensor analysis for physicists , 1955 .

[9]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[10]  G. Arfken.  Mathematical Methods for Physicists , 1967 .

[11]  Jianbo Shi,et al.  Multiclass spectral clustering , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[12]  Stephen Lin,et al.  Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval , 2007, IEEE Transactions on Image Processing.

[13]  Erik W. Grafarend,et al.  Fourth order Taylor–Kármán structured covariance tensor for gravity gradient predictions by means of the Hankel transformation , 2015 .

[14]  Nanda Kambhatla,et al.  Dimension Reduction by Local Principal Component Analysis , 1997, Neural Computation.

[15]  Meixia Tao,et al.  Joint Source and Relay Precoding Designs for MIMO Two-Way Relaying Based on MSE Criterion , 2011, IEEE Transactions on Signal Processing.

[16]  Jan A Snyman,et al.  Practical Mathematical Optimization: An Introduction to Basic Optimization Theory and Classical and New Gradient-Based Algorithms , 2005 .

[17]  Are Hjørungnes,et al.  Complex-Valued Matrix Differentiation: Techniques and Key Results , 2007, IEEE Transactions on Signal Processing.

[18]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 1991 .

[19]  Charles W. Wampler,et al.  Manipulator Inverse Kinematic Solutions Based on Vector Formulations and Damped Least-Squares Methods , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[20]  Jerry D. Gibson,et al.  Handbook of Image and Video Processing , 2000 .

[21]  Emre Telatar,et al.  Capacity of Multi-antenna Gaussian Channels , 1999, Eur. Trans. Telecommun..

[22]  Masaaki Harada.  Extremal Type I Zk-Codes and k-Frames of Odd Unimodular Lattices , 2015, IEEE Trans. Inf. Theory.

[23]  Peter Donnelly,et al.  Superfamilies of Evolved and Designed Networks , 2004 .

[24]  Dit-Yan Yeung,et al.  A Convex Formulation of Modularity Maximization for Community Detection , 2011, IJCAI.

[25]  Jie Sun,et al.  Controllability transition and nonlocality in network control. , 2013, Physical review letters.

[26]  James Serrin,et al.  Mathematical Principles of Classical Fluid Mechanics , 1959 .

[27]  Chien Chern Cheah,et al.  Topology-Based Controllability Problem in Network Systems , 2017, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[28]  Chenguang Yang,et al.  Model Predictive Control of Nonholonomic Chained Systems Using General Projection Neural Networks Optimization , 2015, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[29]  Robert R. Christian,et al.  Organizing and understanding a winter's seagrass foodweb network through effective trophic levels , 1999 .

[30]  Isaac S. Klickstein,et al.  Optimal control of complex networks: Balancing accuracy and energy of the control action. , 2017, Chaos.

[31]  Wei Xing Zheng,et al.  Iterative identification of block-oriented nonlinear systems based on biconvex optimization , 2015, Syst. Control. Lett..

[32]  M E J Newman,et al.  Modularity and community structure in networks. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[33]  Jonathan H. Manton,et al.  Optimization algorithms exploiting unitary constraints , 2002, IEEE Trans. Signal Process..

[34]  K. Wiberg,et al.  Hartree–Fock second derivatives and electric field properties in a solvent reaction field: Theory and application , 1991 .

[35]  X. Rong Li,et al.  Robust linear estimation fusion with allowable unknown cross-covariance , 2014, 17th International Conference on Information Fusion (FUSION).

[36]  Wei Dai,et al.  Convergence of Gradient Descent for Low-Rank Matrix Approximation , 2015, IEEE Transactions on Information Theory.

[37]  Henk A. L. Kiers,et al.  Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems , 2002, Comput. Stat. Data Anal..

[38]  Guoqi Li,et al.  Minimum-cost control of complex networks , 2015 .

[39]  Biing-Hwang Juang,et al.  Discriminative learning for minimum error classification [pattern recognition] , 1992, IEEE Trans. Signal Process..

[40]  O. Simeone,et al.  Towards a joint optimization of scheduling and beamforning for MIMO downlink , 2006, 2006 IEEE Ninth International Symposium on Spread Spectrum Techniques and Applications.

[41]  Ali H. Sayed,et al.  Multiuser Two-Way Amplify-and-Forward Relay Processing and Power Control Methods for Beamforming Systems , 2010, IEEE Transactions on Signal Processing.

[42]  Milan Sonka,et al.  Image Processing, Analysis and Machine Vision , 1993, Springer US.

[43]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[44]  J. Perdew,et al.  Density-Functional Theory for Fractional Particle Number: Derivative Discontinuities of the Energy , 1982 .

[45]  Derek Ruths,et al.  Control Profiles of Complex Networks , 2014, Science.

[46]  R. Aris.  Vectors, Tensors and the Basic Equations of Fluid Mechanics , 1962 .

[47]  Jie Ding,et al.  Optimal control of complex networks based on matrix differentiation , 2016 .

[48]  Wei Yu,et al.  Iterative water-filling for Gaussian vector multiple-access channels , 2001, IEEE Transactions on Information Theory.

[49]  Xuelong Li,et al.  Patch Alignment for Dimensionality Reduction , 2009, IEEE Transactions on Knowledge and Data Engineering.

[50]  D. Ward,et al.  Directional Derivative Calculus and Optimality Conditions in Nonsmooth Mathematical Programming , 1989 .

[51]  Jie Ren,et al.  Controlling complex networks: How much energy is needed? , 2012, Physical review letters.

[52]  Katherine K. Perkins,et al.  Observations on student difficulties with mathematics in upper-division electricity and magnetism , 2012 .

[53]  Hongyu Zhao,et al.  Normalized modularity optimization method for community identification with degree adjustment. , 2013, Physical review. E, Statistical, nonlinear, and soft matter physics.

[54]  Qi Hao,et al.  Mobile Target Scenario Recognition Via Low-Cost Pyroelectric Sensing System: Toward a Context-Enhanced Accurate Identification , 2014, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[55]  Duncan J. Watts,et al.  Collective dynamics of ‘small-world’ networks , 1998, Nature.

[56]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[57]  I. Dhillon,et al.  A Unified View of Kernel k-means , Spectral Clustering and Graph Cuts , 2004 .

[58]  Xuelong Li,et al.  A Unified Learning Framework for Single Image Super-Resolution , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[59]  Albert-László Barabási,et al.  Controllability of complex networks , 2011, Nature.

[60]  Min Han,et al.  A fully automatic ocular artifact removal from EEG based on fourth-order tensor method , 2014 .

[61]  A. Bloch,et al.  A variational problem on Stiefel manifolds , 2006, math/0609038.