Reproducing kernel Hilbert C*-module and kernel mean embeddings

Kernel methods have been among the most popular techniques in machine learning, where learning tasks are solved using the property of reproducing kernel Hilbert space (RKHS). In this paper, we propose a novel data analysis framework with reproducing kernel Hilbert C∗-module (RKHM) and kernel mean embedding (KME) in RKHM. Since RKHM contains richer information than RKHS or vector-valued RKHS (vvRKHS), analysis with RKHM enables us to capture and extract structural properties in such as functional data. We show a branch of theories for RKHM to apply to data analysis, including the representer theorem, c ©2021 Yuka Hashimoto, Isao Ishikawa, Masahiro Ikeda, Fuyuta Komura, Takeshi Katsura, and Yoshinobu Kawahara. License: CC-BY 4.0, see https://creativecommons.org/licenses/by/4.0/. Attribution requirements are provided at http://jmlr.org/papers/v22/20-1346.html. Hashimoto, Ishikawa, Ikeda, Komura, Katsura, and Kawahara and the injectivity and universality of the proposed KME. We also show RKHM generalizes RKHS and vvRKHS. Then, we provide concrete procedures for employing RKHM and the proposed KME to data analysis.

[1]  RakotomamonjyAlain,et al.  Operator-valued kernels for learning from functional response data , 2016 .

[2]  E. Christopher Lance,et al.  Hilbert C*-Modules: Stabilisation or absorption , 1995 .

[3]  J. Diestel.  Sequences and series in Banach spaces , 1984 .

[4]  George Michailidis,et al.  Operator-valued kernel-based vector autoregressive models for network inference , 2014, Machine Learning.

[5]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[6]  Keisuke Fujii,et al.  Dynamic mode decomposition in vector-valued reproducing kernel Hilbert spaces for extracting dynamical structure among observables , 2018, Neural Networks.

[7]  K SriperumbudurBharath,et al.  Universality, Characteristic Kernels and RKHS Embedding of Measures , 2011 .

[8]  George Smyrlis,et al.  Local convergence of the steepest descent method in Hilbert spaces , 2004 .

[9]  Shiqi Wang,et al.  Heterogeneous Domain Adaptation via Nonlinear Matrix Factorization , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[10]  齋藤 三郎,et al.  Theory of reproducing kernels and its applications , 1988 .

[11]  S. Rachev.  On a Class of Minimal Functionals on a Space of Probability Measures , 1985 .

[12]  Yoshinobu Kawahara,et al.  Dynamic Mode Decomposition with Reproducing Kernels for Koopman Spectral Analysis , 2016, NIPS.

[13]  P. Rebentrost,et al.  Quantum machine learning for quantum anomaly detection , 2017, 1710.07405.

[14]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[15]  Steven L. Brunton,et al.  Deep learning for universal linear embeddings of nonlinear dynamics , 2017, Nature Communications.

[16]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[17]  Yunfei Ye.  The Matrix Hilbert Space and Its Application to Matrix Learning , 2017, 1706.08110.

[18]  Naoya Takeishi,et al.  Learning Koopman Invariant Subspaces for Dynamic Mode Decomposition , 2017, NIPS.

[19]  A. Müller.  Integral Probability Metrics and Their Generating Classes of Functions , 1997, Advances in Applied Probability.

[20]  J. Diestel,et al.  On vector measures , 1974 .

[21]  J. Powell.  Mathematical Methods in Physics , 1965 .

[22]  Prasenjit Deb,et al.  Geometry of quantum state space and quantum correlations , 2015, Quantum Inf. Process..

[23]  Murphy's {\em Positive definite kernels and Hilbert C${}^*$--modules} reorganized , 2009, 0906.5408.

[24]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[25]  Naoya Takeishi,et al.  Subspace dynamic mode decomposition for stochastic Koopman analysis. , 2017, Physical review. E.

[26]  Bernhard Schölkopf,et al.  Hilbert Space Embeddings and Metrics on Probability Measures , 2009, J. Mach. Learn. Res..

[27]  Le Song,et al.  A Hilbert Space Embedding for Distributions , 2007, Discovery Science.

[28]  Gert R. G. Lanckriet,et al.  On the empirical estimation of integral probability metrics , 2012 .

[29]  Bernhard Schölkopf,et al.  Kernel Measures of Conditional Dependence , 2007, NIPS.

[30]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[31]  N. Dinculeanu.  Vector Integration and Stochastic Integration in Banach Spaces , 2000, Oxford Handbooks Online.

[32]  Daniel R. Terno,et al.  Quantum Information and Relativity Theory , 2002, quant-ph/0212023.

[33]  S. Itoh.  REPRODUCING KERNELS IN MODULES OVER C^*-ALGEBRAS AND THEIR APPLICATIONS , 1990 .

[34]  Igor Mezic,et al.  Koopman Operator Spectrum for Random Dynamical Systems , 2017, Journal of Nonlinear Science.

[35]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[36]  Ichiro Takeuchi,et al.  Selective Inference for Sparse High-Order Interaction Models , 2017, ICML.

[37]  Reproducing kernel Hilbert C∗-modules and kernels associated with cocycles , 2008 .

[38]  Vittorio Murino,et al.  A Unifying Framework in Vector-valued Reproducing Kernel Hilbert Spaces for Manifold Regularization and Co-Regularized Multi-view Learning , 2014, J. Mach. Learn. Res..

[39]  Keisuke Fujii,et al.  Metric on Nonlinear Dynamical Systems with Perron-Frobenius Operators , 2018, NeurIPS.

[40]  Esma Balkr,et al.  Using Density Matrices in a Compositional Distributional Model of Meaning , 2014 .

[41]  Hans-Georg Müller,et al.  Functional Data Analysis , 2016 .

[42]  Stefan Klus,et al.  Eigendecompositions of Transfer Operators in Reproducing Kernel Hilbert Spaces , 2017, J. Nonlinear Sci..

[43]  E. Troitsky,et al.  HilbertC*- andW*-modules and their morphisms , 2000 .

[44]  Yoshinobu Kawahara,et al.  Krylov Subspace Method for Nonlinear Dynamical Systems with Random Noise , 2019, J. Mach. Learn. Res..

[45]  L. Ballentine,et al.  Probabilistic and Statistical Aspects of Quantum Theory , 1982 .

[46]  Bernhard Schölkopf,et al.  Kernel Mean Embedding of Distributions: A Review and Beyonds , 2016, Found. Trends Mach. Learn..

[47]  Ingo Steinwart,et al.  On the Influence of the Kernel on the Consistency of Support Vector Machines , 2002, J. Mach. Learn. Res..

[48]  Bernhard Schölkopf,et al.  Kernel Mean Matching for Content Addressability of GANs , 2019, ICML.

[49]  M. Urner.  Scattered Data Approximation , 2016 .

[50]  J. Ramsay,et al.  Introduction to Functional Data Analysis , 2007 .

[51]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .