An Integration-Based Approach to Pattern Clustering and Classification

Methods based on information theory, such as the Relevance Index (RI), have been employed to study complex systems for their ability to detect significant groups of variables, well integrated among one another and well separated from the others, which provide a functional block description of the system under analysis. The integration (or zI in its standardized form) is a metric that can express the significance of a group of variables for the system under consideration: the higher the zI, the more significant the group. In this paper, we use this metric for an unusual application to a pattern clustering and classification problem. The results show that the centroids of the clusters of patterns identified by the method are effective for distance-based classification algorithms. We compare such a method with other conventional classification approaches to highlight its main features and to address future research towards the refinement of its accuracy and computational efficiency.

[1]  Marco Fiorucci,et al.  The Search for Candidate Relevant Subsets of Variables in Complex Systems , 2015, Artificial Life.

[2]  Michele Amoretti,et al.  Efficient Search of Relevant Structures in Complex Systems , 2016, AI*IA.

[3]  Jacob Goldberger,et al.  Nonparametric Information Theoretic Clustering Algorithm , 2010, ICML.

[4]  Michele Amoretti,et al.  Searching Relevant Variable Subsets in Complex Systems Using K-Means PSO , 2017, WIVACE.

[5]  Sebastian Nowozin,et al.  Information Theoretic Clustering Using Minimum Spanning Trees , 2012, DAGM/OAGM Symposium.

[6]  Fei Sha,et al.  Demystifying Information-Theoretic Clustering , 2013, ICML.

[7]  Michele Amoretti,et al.  GPU-Based Parallel Search of Relevant Variable Sets in Complex Systems , 2016, WIVACE.

[8]  Ángel Fernando Kuri Morales,et al.  A Clustering Method Based on the Maximum Entropy Principle , 2015, Entropy.

[9]  G. Edelman,et al.  A measure for brain complexity: relating functional segregation and integration in the nervous system. , 1994, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Teuvo Kohonen,et al.  Learning vector quantization , 1998 .

[11]  Michele Amoretti,et al.  An Iterative Information-Theoretic Approach to the Detection of Structures in Complex Systems , 2018, Complex..

[12]  Michele Amoretti,et al.  A Relevance Index Method to Infer Global Properties of Biological Networks , 2017, WIVACE.

[13]  Stefano Cagnoni,et al.  OSLVQ: a training strategy for optimum-size learning vector quantization classifiers , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[14]  Meihong Wang,et al.  Information Theoretical Clustering via Semidefinite Programming , 2011, AISTATS.

[15]  Marco Fiorucci,et al.  Exploring the organisation of complex systems through the dynamical interactions among their relevant subsets , 2015, ECAL.

[16]  D. P. Russell,et al.  Functional Clustering: Identifying Strongly Interactive Brain Regions in Neuroimaging Data , 1998, NeuroImage.