Single Independent Component Recovery and Applications

Latent variable discovery is a central problem in data analysis with a broad range of applications in applied science. In this work, we consider data given as an invertible mixture of two statistically independent components, and assume that one of the components is observed while the other is hidden. Our goal is to recover the hidden component. For this purpose, we propose an autoencoder equipped with a discriminator. Unlike the standard nonlinear ICA problem, which was shown to be non-identifiable, in the special case of ICA we consider here, we show that our approach can recover the component of interest up to entropy-preserving transformation. We demonstrate the performance of the proposed approach on several datasets, including image synthesis, voice cloning, and fetal ECG extraction.

[1]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[2]  Radek Martinek,et al.  Adaptive Linear Neuron for Fetal Electrocardiogram Extraction , 2018, 2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom).

[3]  Aapo Hyvärinen,et al.  Nonlinear ICA Using Auxiliary Variables and Generalized Contrastive Learning , 2018, AISTATS.

[4]  Yoshua Bengio,et al.  MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesis , 2019, NeurIPS.

[5]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[6]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[7]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[8]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[9]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[10]  Lior Wolf,et al.  TTS Skins: Speaker Conversion via ASR , 2019, INTERSPEECH.

[11]  Christopher Burgess,et al.  beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework , 2016, ICLR 2016.

[12]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[13]  Lior Wolf,et al.  Unsupervised Singing Voice Conversion , 2019, INTERSPEECH.

[14]  Jeff A. Bilmes,et al.  Deep Canonical Correlation Analysis , 2013, ICML.

[15]  Aapo Hyvärinen,et al.  Variational Autoencoders and Nonlinear ICA: A Unifying Framework , 2019, AISTATS.

[16]  B. Widrow,et al.  Adaptive noise cancelling: Principles and applications , 1975 .

[17]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[18]  Junichi Yamagishi,et al.  CSTR VCTK Corpus: English Multi-speaker Corpus for CSTR Voice Cloning Toolkit , 2017 .

[19]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[20]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[21]  Andriy Mnih,et al.  Disentangling by Factorising , 2018, ICML.

[22]  Yoshua Bengio,et al.  Learning Independent Features with Adversarial Nets for Non-linear ICA , 2017, 1710.05050.

[23]  Boris Ginsburg,et al.  Jasper: An End-to-End Convolutional Neural Acoustic Model , 2019, INTERSPEECH.

[24]  Herbert Jaeger,et al.  The''echo state''approach to analysing and training recurrent neural networks , 2001 .

[25]  Jeffrey M. Hausdorff,et al.  Physionet: Components of a New Research Resource for Complex Physiologic Signals". Circu-lation Vol , 2000 .

[26]  Jan Nedoma,et al.  Non-Invasive Fetal Monitoring: A Maternal Surface ECG Electrode Placement-Based Novel Approach for Optimization of Adaptive Filter Control Parameters Using the LMS and RLS Algorithms , 2017, Sensors.

[27]  Abhishek Kumar,et al.  Variational Inference of Disentangled Latent Concepts from Unlabeled Observations , 2017, ICLR.

[28]  Aapo Hyvärinen,et al.  Nonlinear ICA of Temporally Dependent Stationary Sources , 2017, AISTATS.

[29]  Roy R. Lederman,et al.  Learning the geometry of common latent variables using alternating-diffusion , 2015 .

[30]  R. Sameni,et al.  A non-invasive multimodal foetal ECG–Doppler dataset for antenatal cardiology research , 2021, Scientific data.

[31]  MarchandMario,et al.  Domain-adversarial training of neural networks , 2016 .

[32]  Uri Shaham,et al.  Batch Effect Removal via Batch-Free Encoding , 2018, bioRxiv.

[33]  Karl Pearson F.R.S..  LIII. On lines and planes of closest fit to systems of points in space , 1901 .

[34]  Aapo Hyvärinen,et al.  Unsupervised Feature Extraction by Time-Contrastive Learning and Nonlinear ICA , 2016, NIPS.

[35]  Yanjun Zeng,et al.  Research of fetal ECG extraction using wavelet analysis and adaptive filtering , 2013, Comput. Biol. Medicine.

[36]  Lior Wolf,et al.  A Universal Music Translation Network , 2018, ICLR.

[37]  Laurenz Wiskott,et al.  An extension of slow feature analysis for nonlinear blind source separation , 2014, J. Mach. Learn. Res..

[38]  Bernhard Schölkopf,et al.  Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations , 2018, ICML.

[39]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[40]  U. Rajendra Acharya,et al.  An efficient compression of ECG signals using deep convolutional autoencoders , 2018, Cognitive Systems Research.

[41]  Guillaume Desjardins,et al.  Understanding disentangling in $\beta$-VAE , 2018, 1804.03599.

[42]  Zhuo Chen,et al.  ESPnet-SE: End-To-End Speech Enhancement and Separation Toolkit Designed for ASR Integration , 2020, 2021 IEEE Spoken Language Technology Workshop (SLT).

[43]  Yochai Blau,et al.  The Perception-Distortion Tradeoff , 2017, CVPR.

[44]  Guillaume Lample,et al.  Fader Networks: Manipulating Images by Sliding Attributes , 2017, NIPS.

[45]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[46]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[47]  Uri Shaham,et al.  Learning by coincidence: Siamese networks and common variable learning , 2018, Pattern Recognit..

[48]  Radek Martinek,et al.  A Review of Signal Processing Techniques for Non-Invasive Fetal Electrocardiography , 2020, IEEE Reviews in Biomedical Engineering.

[49]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[50]  Bernhard Schölkopf,et al.  Measuring Statistical Dependence with Hilbert-Schmidt Norms , 2005, ALT.

[51]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[52]  Ullrich Köthe,et al.  Disentanglement by Nonlinear ICA with General Incompressible-flow Networks (GIN) , 2020, ICLR.

[53]  Visa Koivunen,et al.  Identifiability, separability, and uniqueness of linear ICA models , 2004, IEEE Signal Processing Letters.

[54]  Karen Livescu,et al.  Nonparametric Canonical Correlation Analysis , 2015, ICML.

[55]  R. Swarnalath,et al.  A Novel Technique for Extraction of FECG using Multi Stage Adaptive Filtering , 2010 .

[56]  Pieter Abbeel,et al.  InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets , 2016, NIPS.

[57]  Roger B. Grosse,et al.  Isolating Sources of Disentanglement in Variational Autoencoders , 2018, NeurIPS.

[58]  Dana H. Brooks,et al.  Structured Disentangled Representations , 2018, AISTATS.

[59]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[60]  Ricardo Gutierrez-Osuna,et al.  Group Latent Embedding for Vector Quantized Variational Autoencoder in Non-Parallel Voice Conversion , 2019, INTERSPEECH.

[61]  Bernhard Schölkopf,et al.  The Incomplete Rosetta Stone problem: Identifiability results for Multi-view Nonlinear ICA , 2019, UAI.

[62]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.