Clustering versus Incremental Learning Multi-Codebook Fuzzy Neural Network for Multi-Modal Data Classification

One of the challenges in machine learning is a classification in multi-modal data. The problem needs a customized method as the data has a feature that spreads in several areas. This study proposed a multi-codebook fuzzy neural network classifiers using clustering and incremental learning approaches to deal with multi-modal data classification. The clustering methods used are K-Means and GMM clustering. Experiment result, on a synthetic dataset, the proposed method achieved the highest performance with 84.76% accuracy. Whereas on the benchmark dataset, the proposed method has the highest performance with 79.94% accuracy. The proposed method has 24.9% and 4.7% improvements in synthetic and benchmark datasets respectively compared to the original version. The proposed classifier has better accuracy compared to a popular neural network with 10% and 4.7% margin in synthetic and benchmark dataset respectively.

[1]  Stefan Wermter,et al.  Emergence of multimodal action representations from neural network self-organization , 2017, Cognitive Systems Research.

[2]  Sergio Escalera,et al.  Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Erik Cambria,et al.  Towards an intelligent framework for multimodal affective data analysis , 2015, Neural Networks.

[4]  W. Jatmiko,et al.  Fuzzy learning vector quantization particle swarm optimization (FLVQ-PSO) and fuzzy neuro generalized learning vector quantization (FN-GLVQ) for automatic early detection system of heart diseases based on real-time electrocardiogram , 2012, 2012 Proceedings of SICE Annual Conference (SICE).

[5]  Ignazio Gallo,et al.  Multimodal Classification Fusion in Real-World Scenarios , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[6]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[7]  Boris Mirkin,et al.  Clustering For Data Mining: A Data Recovery Approach (Chapman & Hall/Crc Computer Science) , 2005 .

[8]  Daoqiang Zhang,et al.  Multimodal classification of Alzheimer's disease and mild cognitive impairment , 2011, NeuroImage.

[9]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[10]  Atsushi Sato,et al.  Generalized Learning Vector Quantization , 1995, NIPS.

[11]  Mohammad Soleymani,et al.  A survey of multimodal sentiment analysis , 2017, Image Vis. Comput..

[12]  Amir-Masoud Eftekhari-Moghadam,et al.  Multimodal feature extraction and fusion for semantic mining of soccer video: a survey , 2012, Artificial Intelligence Review.

[13]  Mongi A. Abidi,et al.  Survey and analysis of multimodal sensor planning and integration for wide area surveillance , 2009, CSUR.

[14]  Sterling C. Johnson,et al.  Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer's Disease using structural MR and FDG-PET images , 2017, ArXiv.

[15]  Chee Kheong Siew,et al.  Extreme learning machine: Theory and applications , 2006, Neurocomputing.

[16]  Wisnu Jatmiko,et al.  Fuzzy-neuro LVQ and its comparison with fuzzy algorithm LVQ in artificial odor discrimination system. , 2002, ISA transactions.

[17]  Gabriele Moser,et al.  Multimodal Classification of Remote Sensing Images: A Review and Future Directions , 2015, Proceedings of the IEEE.

[18]  Louis-Philippe Morency,et al.  Multimodal Machine Learning: A Survey and Taxonomy , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Tomas Mikolov,et al.  Efficient Large-Scale Multi-Modal Classification , 2018, AAAI.

[20]  Teuvo Kohonen,et al.  Improved versions of learning vector quantization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[21]  Bruce W. Suter,et al.  The multilayer perceptron as an approximation to a Bayes optimal discriminant function , 1990, IEEE Trans. Neural Networks.

[22]  Adrian Preda,et al.  Multimodal and Multiscale Deep Neural Networks for the Early Diagnosis of Alzheimer’s Disease using structural MR and FDG-PET images , 2018, Scientific Reports.

[23]  Matthias Rädle,et al.  Incremental Learning with SVM for Multimodal Classification of Prostatic Adenocarcinoma , 2014, PloS one.

[24]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[25]  Stanislas Chambon,et al.  A Deep Learning Architecture for Temporal Sleep Stage Classification Using Multivariate and Multimodal Time Series , 2017, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  Michel Vacher,et al.  SVM-Based Multimodal Classification of Activities of Daily Living in Health Smart Homes: Sensors, Algorithms, and First Experimental Results , 2010, IEEE Transactions on Information Technology in Biomedicine.

[27]  Mathias Benedek,et al.  Real-Time Multimodal Classification of Internal and External Attention , 2019, ICMI.

[28]  Wisnu Jatmiko,et al.  Arrhytmia classification using Fuzzy-Neuro Generalized Learning Vector Quantization , 2011, 2011 International Conference on Advanced Computer Science and Information Systems.

[29]  Andrés Ortiz,et al.  Ensembles of Deep Learning Architectures for the Early Diagnosis of the Alzheimer's Disease , 2016, Int. J. Neural Syst..

[30]  David Dagan Feng,et al.  Content-Based Medical Image Retrieval: A Survey of Applications to Multidimensional and Multimodality Data , 2013, Journal of Digital Imaging.

[31]  D. N. Geary.  Mixture Models: Inference and Applications to Clustering , 1989 .

[32]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[33]  Aprinaldi,et al.  Multi codebook LVQ-based artificial neural network using clustering approach , 2015, 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS).