Deep Goal-Oriented Clustering

Clustering and prediction are two primary tasks in the fields of unsupervised and supervised learning, respectively. Although much of the recent advances in machine learning have been centered around those two tasks, the interdependent, mutually beneficial relationship between them is rarely explored. One could reasonably expect appropriately clustering the data would aid the downstream prediction task and, conversely, a better prediction performance for the downstream task could potentially inform a more appropriate clustering strategy. In this work, we focus on the latter part of this mutually beneficial relationship. To this end, we introduce Deep Goal-Oriented Clustering (DGC), a probabilistic framework that clusters the data by jointly using supervision via side-information and unsupervised modeling of the inherent data structure in an end-to-end fashion. We show the effectiveness of our model on a range of datasets by achieving prediction accuracies comparable to the state-of-the-art, while, more importantly in our setting, simultaneously learning congruent clustering strategies.

[1]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[2]  I JordanMichael,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008 .

[3]  Eric Bair,et al.  Semi‐supervised clustering methods , 2013, Wiley interdisciplinary reviews. Computational statistics.

[4]  Gang Wang,et al.  Reinforced Similarity Integration in Image-Rich Information Networks , 2013, IEEE Transactions on Knowledge and Data Engineering.

[5]  Francesco G. B. De Natale,et al.  Classtering: Joint Classification and Clustering with Mixture of Factor Analysers , 2016, ECAI.

[6]  Huachun Tan,et al.  Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering , 2016, IJCAI.

[7]  Thorsten Joachims,et al.  Supervised clustering with support vector machines , 2005, ICML.

[8]  Ali Farhadi,et al.  Unsupervised Deep Embedding for Clustering Analysis , 2015, ICML.

[9]  Matthijs Douze,et al.  Deep Clustering for Unsupervised Learning of Visual Features , 2018, ECCV.

[10]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[11]  Graham W. Taylor,et al.  Improved Regularization of Convolutional Neural Networks with Cutout , 2017, ArXiv.

[12]  Tom Ronan,et al.  Avoiding common pitfalls when clustering biological data , 2016, Science Signaling.

[13]  Joachim M. Buhmann,et al.  Nonparametric Bayesian Image Segmentation , 2008, International Journal of Computer Vision.

[14]  Raquel Urtasun,et al.  Deep Spectral Clustering Learning , 2017, ICML.

[15]  Ismail Uysal,et al.  Learning Latent Representations in Neural Networks for Clustering through Pseudo Supervision and Graph-based Activity Regularization , 2018, ICLR.

[16]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Xiaojin Zhu,et al.  Semi-Supervised Learning , 2010, Encyclopedia of Machine Learning.

[19]  Max Welling,et al.  Semi-supervised Learning with Deep Generative Models , 2014, NIPS.

[20]  A. Zimek,et al.  On Using Class-Labels in Evaluation of Clusterings , 2010 .

[21]  Ian Davidson,et al.  Flexible constrained spectral clustering , 2010, KDD.

[22]  Michael I. Jordan Graphical Models , 2003 .

[23]  Andrew Y. Ng,et al.  Reading Digits in Natural Images with Unsupervised Feature Learning , 2011 .

[24]  Ronen Basri,et al.  SpectralNet: Spectral Clustering using Deep Neural Networks , 2018, ICLR.

[25]  Michael I. Jordan,et al.  Graphical Models, Exponential Families, and Variational Inference , 2008, Found. Trends Mach. Learn..

[26]  Daniel Khashabi,et al.  Clustering With Side Information: From a Probabilistic Model to a Deterministic Algorithm , 2015, ArXiv.

[27]  Claire Cardie,et al.  Clustering with Instance-Level Constraints , 2000, AAAI/IAAI.

[28]  Khaled Shaalan,et al.  Speech Recognition Using Deep Neural Networks: A Systematic Review , 2019, IEEE Access.

[29]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.