Deep Clustering with Spherical Distance in Latent Space

This paper studies the problem of deep joint-clustering using auto-encoder. For this task, most algorithms solve a multi-objective optimization problem, where it is then transformed into a sing-objective problem by linear scalarization techniques. However, it introduces the scaling problem in latent space in a class of algorithms. We propose an extension to solve this problem by using scale invariance distance functions. The advantage of this extension is demonstrated for a particular case of joint-clustering with MSSC (minimizing sum-of-squares clustering). Numerical experiments on several benchmark datasets illustrate the superiority of our extension over state-of-the-art algorithms with respect to clustering accuracy.

[1]  Jiawei Han,et al.  Locally Consistent Concept Factorization for Document Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[2]  Wei Wang,et al.  Deep Embedding Network for Clustering , 2014, 2014 22nd International Conference on Pattern Recognition.

[3]  Vipin Kumar,et al.  The Challenges of Clustering High Dimensional Data , 2004 .

[4]  Maoguo Gong,et al.  DRLnet: Deep Difference Representation Learning Network and An Unsupervised Optimization Framework , 2017, IJCAI.

[5]  En Zhu,et al.  Deep Embedded Clustering with Data Augmentation , 2018, ACML.

[6]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[7]  Cheng Deng,et al.  Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Francesco Cricri,et al.  Clustering and Unsupervised Anomaly Detection with l2 Normalized Deep Auto-Encoder Representations , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[9]  Jianping Yin,et al.  Improved Deep Embedded Clustering with Local Structure Preservation , 2017, IJCAI.

[10]  Inderjit S. Dhillon,et al.  Concept Decompositions for Large Sparse Text Data Using Clustering , 2004, Machine Learning.

[11]  Hongdong Li,et al.  Neural Collaborative Subspace Clustering , 2019, ICML.

[12]  Feng Liu,et al.  Auto-encoder Based Data Clustering , 2013, CIARP.

[13]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[14]  Roland Vollgraf,et al.  Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms , 2017, ArXiv.

[15]  Ian D. Reid,et al.  Scalable Deep k-Subspace Clustering , 2018, ACCV.

[16]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[17]  Lei Luo,et al.  Deep Discriminative Clustering Network , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[18]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[19]  Mohamed Nadif,et al.  Spectral Clustering via Ensemble Deep Autoencoder Learning (SC-EDAE) , 2019, Pattern Recognit..

[20]  Dhruv Batra,et al.  Joint Unsupervised Learning of Deep Representations and Image Clusters , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Vladlen Koltun,et al.  Deep Continuous Clustering , 2018, ArXiv.

[22]  Jiancheng Lv,et al.  Unsupervised Multi-Manifold Clustering by Learning Deep Representation , 2017, AAAI Workshops.

[23]  Dipanjan Das,et al.  Deep Representation Learning Characterized by Inter-Class Separation for Image Clustering , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[24]  Shuigeng Zhou,et al.  DeepCluster: A General Clustering Framework Based on Deep Learning , 2017, ECML/PKDD.

[25]  Tong Zhang,et al.  Deep Subspace Clustering Networks , 2017, NIPS.

[26]  Huu Le,et al.  DeepVQ: A Deep Network Architecture for Vector Quantization , 2018, CVPR Workshops.

[27]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[28]  Bo Yang,et al.  Towards K-means-friendly Spaces: Simultaneous Deep Learning and Clustering , 2016, ICML.

[29]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[30]  Matthijs Douze,et al.  Deep Clustering for Unsupervised Learning of Visual Features , 2018, ECCV.