NN-EVCLUS: Neural Network-based Evidential Clustering

Evidential clustering is an approach to clustering based on the use of Dempster-Shafer mass functions to represent cluster-membership uncertainty. In this paper, we introduce a neural-network based evidential clustering algorithm, called NN-EVCLUS, which learns a mapping from attribute vectors to mass functions, in such a way that more similar inputs are mapped to output mass functions with a lower degree of conflict. The neural network can be paired with a one-class support vector machine to make it robust to outliers and allow for novelty detection. The network is trained to minimize the discrepancy between dissimilarities and degrees of conflict for all or some object pairs. Additional terms can be added to the loss function to account for pairwise constraints or labeled data, which can also be used to adapt the metric. Comparative experiments show the superiority of N-EVCLUS over state-of-the-art evidential clustering algorithms for a range of unsupervised and constrained clustering tasks involving both attribute and dissimilarity data.

[1]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[2]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[3]  Masashi Sugiyama,et al.  Local Fisher discriminant analysis for supervised dimensionality reduction , 2006, ICML.

[4]  Sean Hughes,et al.  Clustering by Fast Search and Find of Density Peaks , 2016 .

[5]  Quan Pan,et al.  ECMdd: Evidential c-medoids clustering with multiple prototypes , 2016, Pattern Recognit..

[6]  Jean Dezert,et al.  Credal c-means clustering method based on belief functions , 2015, Knowl. Based Syst..

[7]  Luís B. Almeida,et al.  Speeding up Backpropagation , 1990 .

[8]  Jian Yang,et al.  Essence of kernel Fisher discriminant: KPCA plus LDA , 2004, Pattern Recognit..

[9]  M. Cugmas,et al.  On comparing partitions , 2015 .

[10]  Andrew R. Webb,et al.  Multidimensional scaling by iterative majorization using radial basis functions , 1995, Pattern Recognit..

[11]  Georg Peters,et al.  Some refinements of rough k-means clustering , 2006, Pattern Recognit..

[12]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[13]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[14]  Thierry Denoeux Calibrated model-based evidential clustering using bootstrapping , 2020, Inf. Sci..

[15]  Kuo-Lung Wu,et al.  Unsupervised possibilistic clustering , 2006, Pattern Recognit..

[16]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Thierry Denoeux,et al.  CEVCLUS: evidential clustering with instance-level constraints for relational data , 2014, Soft Comput..

[18]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[19]  Thierry Denoeux,et al.  ECM: An evidential version of the fuzzy c , 2008, Pattern Recognit..

[20]  Marcel J. T. Reinders,et al.  Sign Language Recognition by Combining Statistical DTW and Independent Classification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[22]  Thierry Denoeux,et al.  BPEC: Belief-Peaks Evidential Clustering , 2019, IEEE Transactions on Fuzzy Systems.

[23]  Thierry Denoeux,et al.  EVCLUS: evidential clustering of proximity data , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[24]  James M. Keller,et al.  A possibilistic fuzzy c-means clustering algorithm , 2005, IEEE Transactions on Fuzzy Systems.

[25]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[26]  Shengcai Liao,et al.  Deep Metric Learning for Person Re-identification , 2014, 2014 22nd International Conference on Pattern Recognition.

[27]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[28]  Thierry Denoeux,et al.  k-CEVCLUS: Constrained evidential clustering of large dissimilarity data , 2017, Knowl. Based Syst..

[29]  Cajo J. F. ter Braak,et al.  Approximating a similarity matrix by a latent class model , 2013 .

[30]  Thierry Denoeux,et al.  CECM: Constrained evidential C-means algorithm , 2012, Comput. Stat. Data Anal..

[31]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[32]  W. Peizhuang Pattern Recognition with Fuzzy Objective Function Algorithms (James C. Bezdek) , 1983 .

[33]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[34]  Songsak Sriboonchitta,et al.  Evaluating and Comparing Soft Partitions: An Approach Based on Dempster–Shafer Theory , 2018, IEEE Transactions on Fuzzy Systems.

[35]  Pawan Lingras,et al.  Applying Rough Set Concepts to Clustering , 2012 .

[36]  Quan Pan,et al.  Median evidential c-means algorithm and its application to community detection , 2015, Knowl. Based Syst..

[37]  Richard Weber,et al.  Soft clustering - Fuzzy and rough approaches and their extensions and derivatives , 2013, Int. J. Approx. Reason..

[38]  Quan Pan,et al.  Belief C-Means: An extension of Fuzzy C-Means algorithm in belief functions framework , 2012, Pattern Recognit. Lett..

[39]  Thierry Denoeux,et al.  RECM: Relational evidential c-means algorithm , 2009, Pattern Recognit. Lett..

[40]  Anil K. Jain,et al.  Representation and Recognition of Handwritten Digits Using Deformable Templates , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[41]  Joachim M. Buhmann,et al.  Pairwise Data Clustering by Deterministic Annealing , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[42]  L. Zadeh Fuzzy sets as a basis for a theory of possibility , 1999 .

[43]  Pasi Fränti,et al.  K-means properties on six clustering benchmark datasets , 2018, Applied Intelligence.

[44]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[45]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[46]  James M. Keller,et al.  Fuzzy Models and Algorithms for Pattern Recognition and Image Processing , 1999 .

[47]  Yann LeCun,et al.  Signature Verification Using A "Siamese" Time Delay Neural Network , 1993, Int. J. Pattern Recognit. Artif. Intell..

[48]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[49]  Thierry Denoeux,et al.  Beyond Fuzzy, Possibilistic and Rough: An Investigation of Belief Functions in Clustering , 2016, SMPS.

[50]  Didier Dubois,et al.  Representations of Uncertainty in Artificial Intelligence: Probability and Possibility , 2020, A Guided Tour of Artificial Intelligence Research.

[51]  Thierry Denoeux,et al.  Evidential clustering of large dissimilarity data , 2016, Knowl. Based Syst..

[52]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[53]  Stefanie Seiler,et al.  Finding Groups In Data , 2016 .