Special Issue on Advances in Kernel-Based Learning for Signal Processing

The importance of learning and adaptation in statistical signal processing creates a symbiotic relationship with machine learning. However, the two disciplines possess different momentum and emphasis, which makes it attractive to periodically review trends and new developments in their overlapping spheres of influence. Looking at the recent trends in machine learning, we see increasing interest in kernel methods, Bayesian reasoning, causality, information theoretic learning, reinforcement learning, and nonnumeric data processing, just to name a few. While some of the machine-learning community trends are clearly visible in signal processing, such as the increased popularity of the Bayesian methods and graphical models, others such as kernel approaches are still less prominent. However, kernel methods offer a number of unique advantages for signal processing, and this special issue aims to review some of those. KerneL-BASed LeArnIng: BAcKground

[1]  Bernhard Schölkopf,et al.  A Kernel Method for the Two-Sample-Problem , 2006, NIPS.

[2]  Zaïd Harchaoui,et al.  A regularized kernel-based approach to unsupervised audio segmentation , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Weifeng Liu,et al.  Kernel Adaptive Filtering: A Comprehensive Introduction , 2010 .

[4]  Stéphane Canu,et al.  Estimation of Minimum Measure Sets in Reproducing Kernel Hilbert Spaces and Applications. , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[5]  Zaïd Harchaoui,et al.  A Fast, Consistent Kernel Two-Sample Test , 2009, NIPS.

[6]  Arthur Gretton,et al.  An online support vector machine for abnormal events detection , 2006, Signal Process..

[7]  Paul Over,et al.  Video shot boundary detection: Seven years of TRECVid activity , 2010, Comput. Vis. Image Underst..

[8]  Thomas Kailath,et al.  An RKHS approach to detection and estimation problems- III: Generalized innovations representations and a likelihood-ratio formula , 1972, IEEE Trans. Inf. Theory.

[9]  Thomas Kailath,et al.  RKHS approach to detection and estimation problems-I: Deterministic signals in Gaussian noise , 1971, IEEE Trans. Inf. Theory.

[10]  Yuesheng Xu,et al.  Universal Kernels , 2006, J. Mach. Learn. Res..

[11]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[12]  Takafumi Kanamori,et al.  Density Ratio Estimation in Machine Learning , 2012 .

[13]  Bernhard Schölkopf,et al.  Measuring Statistical Dependence with Hilbert-Schmidt Norms , 2005, ALT.

[14]  Douglas A. Reynolds,et al.  A Tutorial on Text-Independent Speaker Verification , 2004, EURASIP J. Adv. Signal Process..

[15]  O. Cappé,et al.  Retrospective Mutiple Change-Point Estimation with Kernels , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[16]  Larry Wasserman,et al.  All of Nonparametric Statistics (Springer Texts in Statistics) , 2006 .

[17]  S. Kay.  Fundamentals of statistical signal processing: estimation theory , 1993 .

[18]  Takafumi Kanamori,et al.  $f$ -Divergence Estimation and Two-Sample Homogeneity Test Under Semiparametric Density-Ratio Models , 2010, IEEE Transactions on Information Theory.

[19]  Gilles Blanchard,et al.  Statistical properties of Kernel Prinicipal Component Analysis , 2019 .

[20]  R. Brunelli,et al.  A Survey on the Automatic Indexing of Video Data, , 1999, J. Vis. Commun. Image Represent..

[21]  Gunnar Rätsch,et al.  Input space versus feature space in kernel-based methods , 1999, IEEE Trans. Neural Networks.

[22]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[23]  Peter F. Craigmile.  All of Statistics: A Concise Course in Statistical Inference , 2005 .

[24]  Thomas Kailath,et al.  An RKHS approach to detection and estimation problems-II: Gaussian signal detection , 1975, IEEE Trans. Inf. Theory.

[25]  Alexander J. Smola,et al.  Hilbert space embeddings of conditional distributions with applications to dynamical systems , 2009, ICML '09.

[26]  N. H. Anderson,et al.  Two-sample test statistics for measuring discrepancies between two multivariate probability density functions using kernel-based density estimates , 1994 .

[27]  Bernhard Schölkopf,et al.  A Kernel Two-Sample Test , 2012, J. Mach. Learn. Res..

[28]  Matthias Hein,et al.  Hilbertian Metrics and Positive Definite Kernels on Probability Measures , 2005, AISTATS.

[29]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[30]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[31]  Le Song,et al.  A Kernel Statistical Test of Independence , 2007, NIPS.

[32]  Arthur Gretton,et al.  Consistent Nonparametric Tests of Independence , 2010, J. Mach. Learn. Res..

[33]  Samy Bengio,et al.  Automatic Speech and Speaker Recognition: Large Margin and Kernel Methods , 2009 .

[34]  Zaïd Harchaoui,et al.  Testing for Homogeneity with Kernel Fisher Discriminant Analysis , 2007, NIPS.

[35]  Takafumi Kanamori,et al.  Statistical analysis of kernel-based least-squares density-ratio estimation , 2012, Machine Learning.

[36]  Paul Fearnhead,et al.  Exact and efficient Bayesian inference for multiple changepoint problems , 2006, Stat. Comput..

[37]  Zaïd Harchaoui,et al.  Kernel Change-point Analysis , 2008, NIPS.

[38]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[39]  Tony Jebara,et al.  Probability Product Kernels , 2004, J. Mach. Learn. Res..

[40]  Douglas A. Reynolds,et al.  Approaches and applications of audio diarization , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[41]  Jean-Philippe Vert,et al.  Consistency and Convergence Rates of One-Class SVMs and Related Algorithms , 2006, J. Mach. Learn. Res..

[42]  E. Lehmann.  Elements of large-sample theory , 1998 .

[43]  Hao Shen,et al.  Fast Kernel-Based Independent Component Analysis , 2009, IEEE Transactions on Signal Processing.

[44]  Felipe Cucker,et al.  Learning Theory: An Approximation Theory Viewpoint: On the bias–variance problem , 2007 .

[45]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[46]  Arjun K. Gupta,et al.  Parametric Statistical Change Point Analysis , 2000 .

[47]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[48]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[49]  Bernhard Schölkopf,et al.  Injective Hilbert Space Embeddings of Probability Measures , 2008, COLT.

[50]  Martin J. Wainwright,et al.  Estimating Divergence Functionals and the Likelihood Ratio by Convex Risk Minimization , 2008, IEEE Transactions on Information Theory.

[51]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[52]  Kenji Fukumizu,et al.  Statistical Consistency of Kernel Canonical Correlation Analysis , 2007 .

[53]  Stephen E. Fienberg,et al.  Testing Statistical Hypotheses , 2005 .

[54]  Bernhard Schölkopf,et al.  The connection between regularization operators and support vector kernels , 1998, Neural Networks.

[55]  N. Vakhania,et al.  Probability Distributions on Banach Spaces , 1987 .

[56]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[57]  Gaël Richard,et al.  On the Correlation of Automatic Audio and Visual Segmentations of Music Videos , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[58]  Manuel Davy,et al.  An online kernel change detection algorithm , 2005, IEEE Transactions on Signal Processing.

[59]  R. D. Figueiredo.  A generalized Fock space framework for nonlinear system and signal analysis , 1983 .

[60]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[61]  Kenji Fukumizu,et al.  Hypothesis testing using pairwise distances and associated kernels , 2012, ICML.

[62]  Gunnar Rätsch,et al.  Constructing Descriptive and Discriminative Nonlinear Features: Rayleigh Coefficients in Kernel Feature Spaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..