Visualization of the Image Geometric Transformation Group Based on Riemannian Manifold

Geometric transformations of images are the predominant factor, which influences the effectiveness of visual tracking and detection tasks in computer vision. Naturally, although it makes significant sense to grasp the process of image geometric transformations, the numerical relationship of geometric transformations cannot be revealed directly from images themselves. Even if the geometric transformation matrices form the three-dimensional special linear group, $\text {Sl}(3,\mathbb {R})$ group, it is difficult to comprehend the manifold of this invisible visual motion, which resides in the high-dimensional space. Furthermore, the main challenge is the deficiency of analytic expressions of the Riemannian logarithmic map to compute the geodesic distance on the $\text {Sl}(3,\mathbb {R})$ manifold. Facing these issues, this paper comes up with a novel approach to visualize the geometric transformation in images by presenting a new metric, and then, computes a set of coordinate-vectors in the three-dimensional state transition space for visualization using the Riemannian stress majorization. The superiority of the presented framework for visualization, in terms of accuracy and efficiency, is demonstrated through abundant experiments on aerial images and moving objects.

[1]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[2]  J. Leeuw Convergence of the majorization method for multidimensional scaling , 1988 .

[3]  Leon A. Gatys,et al.  Image Style Transfer Using Convolutional Neural Networks , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  D. Marquardt An Algorithm for Least-Squares Estimation of Nonlinear Parameters , 1963 .

[5]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[6]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Michael Werman,et al.  How to put probabilities on homographies , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Tianci Liu,et al.  Joint Normalization and Dimensionality Reduction on Grassmannian: A Generalized Perspective , 2018, IEEE Signal Processing Letters.

[9]  Frank Chongwoo Park,et al.  Geometric Direct Search Algorithms for Image Registration , 2007, IEEE Transactions on Image Processing.

[10]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[11]  Frank Dellaert,et al.  IMU Preintegration on Manifold for Efficient Visual-Inertial Maximum-a-Posteriori Estimation , 2015, Robotics: Science and Systems.

[12]  Kim Marriott,et al.  Stress Majorization with Orthogonal Ordering Constraints , 2005, Graph Drawing.

[13]  Ying Zhu,et al.  Measuring Effective Data Visualization , 2007, ISVC.

[14]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[15]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[16]  Tianci Liu,et al.  Visualization of the geometric transformation group based on the Riemannian metric , 2016, 2016 3rd International Conference on Systems and Informatics (ICSAI).

[17]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[18]  Sylvain Paris,et al.  Deep Photo Style Transfer , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Mariusz Bojarski,et al.  VisualBackProp: Efficient Visualization of CNNs for Autonomous Driving , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Dieter Fox,et al.  SE3-nets: Learning rigid body motion using deep neural networks , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Chen Shi Experimental Comparisons of Semi-Supervised Dimensional Reduction Methods , 2011 .

[23]  Michael Werman,et al.  Affine Invariance Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  J. Xavier,et al.  HESSIAN OF THE RIEMANNIAN SQUARED DISTANCE FUNCTION ON CONNECTED LOCALLY SYMMETRIC SPACES WITH APPLICATIONS , 2006 .

[27]  Peter Meer,et al.  Nonlinear Mean Shift over Riemannian Manifolds , 2009, International Journal of Computer Vision.

[28]  Xilin Chen,et al.  Projection Metric Learning on Grassmann Manifold with Application to Video based Face Recognition , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Satoru Kawai,et al.  An Algorithm for Drawing General Undirected Graphs , 1989, Inf. Process. Lett..

[30]  Yui Man Lui,et al.  Advances in matrix manifolds for computer vision , 2012, Image Vis. Comput..

[31]  Brian C. Lovell,et al.  Extrinsic Methods for Coding and Dictionary Learning on Grassmann Manifolds , 2014, International Journal of Computer Vision.

[32]  Tianci Liu,et al.  Efficient and Robust Direct Image Registration Based on Joint Geometric and Photometric Lie Algebra , 2018, IEEE Transactions on Image Processing.

[33]  W. Rossmann Lie Groups: An Introduction through Linear Groups , 2002 .

[34]  J. Kruskal Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[35]  Bohua Zhan,et al.  Smooth Manifolds , 2021, Arch. Formal Proofs.

[36]  Shiguang Shan,et al.  Log-Euclidean Metric Learning on Symmetric Positive Definite Manifold with Application to Image Set Classification , 2015, ICML.

[37]  H. Karcher Riemannian center of mass and mollifier smoothing , 1977 .

[38]  Luc Van Gool,et al.  Deep Learning on Lie Groups for Skeleton-Based Action Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[40]  Frank Chongwoo Park,et al.  A Geometric Particle Filter for Template-Based Visual Tracking , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[42]  R. Whitaker,et al.  Riemannian Metrics on the Space of Solid Shapes , 2006 .

[43]  W. Eric L. Grimson,et al.  Learning visual flows: A Lie algebraic approach , 2009, CVPR.

[44]  Tianci Liu,et al.  Kernel sparse representation on Grassmann manifolds for visual clustering , 2018 .

[45]  Rama Chellappa,et al.  Group motion segmentation using a Spatio-Temporal Driving Force Model , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Yehuda Koren,et al.  Graph Drawing by Stress Majorization , 2004, GD.

[47]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[48]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[49]  Nicholas Ayache,et al.  Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices , 2007, SIAM J. Matrix Anal. Appl..

[50]  Mehrtash Tafazzoli Harandi,et al.  From Manifold to Manifold: Geometry-Aware Dimensionality Reduction for SPD Matrices , 2014, ECCV.

[51]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[52]  Mehrtash Harandi,et al.  Dimensionality Reduction on SPD Manifolds: The Emergence of Geometry-Aware Methods , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.