Diffusion operators for multimodal data analysis

Abstract In this chapter, we present a Manifold Learning viewpoint on the analysis of data arising from multiple modalities. We assume that the high-dimensional multimodal data lie on underlying low-dimensional manifolds and devise a new data-driven representation that accommodates this inherent structure. Based on diffusion geometry, we present three composite operators, facilitating different aspects of fusion of information from different modalities in different settings. These operators are shown to recover the common structures and the differences between modalities in terms of their intrinsic geometry and allow for the construction of data-driven representations which capture these characteristics. The properties of these operators are demonstrated in four applications: recovery of the common variable in two camera views, shape analysis, foetal heart rate identification and sleep dynamics assessment.

[1]  Colin Fyfe,et al.  Kernel and Nonlinear Canonical Correlation Analysis , 2000, IJCNN.

[2]  Hal Daumé,et al.  A Co-training Approach for Multi-view Spectral Clustering , 2011, ICML.

[3]  A. Fraser.  Hidden Markov Models and Dynamical Systems , 2011 .

[4]  A. Chesson,et al.  The American Academy of Sleep Medicine (AASM) Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specifications , 2007 .

[5]  Chinmay Hegde,et al.  Joint Manifolds for Data Fusion , 2010, IEEE Transactions on Image Processing.

[6]  Uri Shaham,et al.  Learning by coincidence: Siamese networks and common variable learning , 2018, Pattern Recognit..

[7]  Bo Wang,et al.  Unsupervised metric fusion by cross diffusion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Li Su,et al.  Wave-Shape Function Analysis , 2016, 1605.01805.

[9]  Roy R. Lederman,et al.  Learning the geometry of common latent variables using alternating-diffusion , 2015 .

[10]  Ronald R. Coifman,et al.  Audio-Visual Group Recognition Using Diffusion Maps , 2010, IEEE Transactions on Signal Processing.

[11]  Leonidas J. Guibas,et al.  Map-based exploration of intrinsic shape differences and variability , 2013, ACM Trans. Graph..

[12]  Hau-Tieng Wu,et al.  Latent common manifold learning with alternating diffusion: analysis and applications , 2016, Applied and Computational Harmonic Analysis.

[13]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[14]  Ronald R. Coifman,et al.  Diffusion Maps for Signal Processing: A Deeper Look at Manifold-Learning Techniques Based on Kernels and Graphs , 2013, IEEE Signal Processing Magazine.

[15]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[16]  S. Quan,et al.  Rules for scoring respiratory events in sleep: update of the 2007 AASM Manual for the Scoring of Sleep and Associated Events. Deliberations of the Sleep Apnea Definitions Task Force of the American Academy of Sleep Medicine. , 2012, Journal of clinical sleep medicine : JCSM : official publication of the American Academy of Sleep Medicine.

[17]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .

[18]  Yoel Shkolnisky,et al.  Multi-View Kernel Consensus For Data Analysis and Signal Processing , 2016, ArXiv.

[19]  Arie Yeredor,et al.  MultiView Diffusion Maps , 2015, Inf. Fusion.

[20]  Gary Froyland,et al.  Dynamic isoperimetry and the geometry of Lagrangian coherent structures , 2014, 1411.7186.

[21]  Christopher J. C. Burges,et al.  Spectral clustering and transductive learning with multiple views , 2007, ICML '07.

[22]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[23]  Chao Wu,et al.  DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-Channel EEG , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[24]  I. Daubechies,et al.  Synchrosqueezed wavelet transforms: An empirical mode decomposition-like tool , 2011 .

[25]  Hau-Tieng Wu,et al.  Diffuse to fuse EEG spectra - Intrinsic geometry of sleep dynamics for classification , 2018, Biomed. Signal Process. Control..

[26]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[27]  Martin G. Frasch,et al.  Efficient Fetal-Maternal ECG Signal Separation from Two Channel Maternal Abdominal ECG via Diffusion-Based Channel Selection , 2017, Front. Physiol..