Fuzzy unsupervised classification of multivariate time trajectories with the Shannon entropy regularization

Fuzzy unsupervised clustering models based on entropy regularization are suggested in order to classify time-varying data. In particular, in the proposed models, objective functions, which are the sum of two terms, are minimized. The first term is a dynamic generalization of intra-cluster distance, in a fuzzy framework, that takes into account the instantaneous and/or longitudinal features of the time-varying observations (the so-called multivariate time trajectories); in this way, the within cluster dispersion is minimized (maximize the internal cohesion). The second term represents the Shannon entropy measure as applied to fuzzy partitions (entropy regularization); then, a given measure of entropy is maximized or, equivalently, the converse of the entropy is minimized. Overall, the total functional depending on both the previous aspects is optimized. The dynamic fuzzy entropy clustering models have been applied to a meteorological dataset and an empirical comparison with the instantaneous and/or longitudinal fuzzy C-means clustering models has been made.

[1]  M. Jardino,et al.  Unsupervised Non-hierarchical Entropy-based Clustering , 2000 .

[2]  Hidetomo Ichihashi,et al.  Linear fuzzy clustering techniques with missing values and their application to local principal component analysis , 2004, IEEE Transactions on Fuzzy Systems.

[3]  Manoranjan Dash,et al.  Entropy-based fuzzy clustering and fuzzy modeling , 2000, Fuzzy Sets Syst..

[4]  C. Lepădatu,et al.  Information energy and information temperature for molecular systems , 2003 .

[5]  Lotfi A. Zadeh,et al.  Outline of a New Approach to the Analysis of Complex Systems and Decision Processes , 1973, IEEE Trans. Syst. Man Cybern..

[6]  Hans-Hermann Bock,et al.  Advances in data science and classification , 1998 .

[7]  Tom E. Bishop,et al.  Blind Image Restoration Using a Block-Stationary Signal Model , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[8]  Sadaaki Miyamoto,et al.  Fuzzy c-Means and Mixture Distribution Models in the Presence of Noise Clusters , 2002, Int. J. Image Graph..

[9]  J. Bezdek.  Numerical taxonomy with fuzzy sets , 1974 .

[10]  Michel Ménard,et al.  Non-Extensive Thermostatistics and Extreme Physical Information for Fuzzy Clustering , 2004 .

[11]  Pierpaolo D'Urso,et al.  Three-way fuzzy clustering models for LR fuzzy time trajectories , 2003, Comput. Stat. Data Anal..

[12]  K. Kosmelj,et al.  Cross-sectional approach for clustering time varying data , 1990 .

[13]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[14]  Yoshiharu Sato,et al.  On a multicriteria fuzzy Clustering Method for 3-Way Data , 1994, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[15]  José Carlos Príncipe,et al.  A new clustering evaluation function using Renyi's information potential , 2000, 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100).

[16]  Pierpaolo D'Urso,et al.  The Geometric Approach to the Comparison of Multivariate Time Trajectories , 2001 .

[17]  Yi Li,et al.  COOLCAT: an entropy-based algorithm for categorical clustering , 2002, CIKM '02.

[18]  Stephen J. Roberts,et al.  Maximum certainty data partitioning , 2000, Pattern Recognit..

[19]  W. Bossert,et al.  The Measurement of Diversity , 2001 .

[20]  Yi Zhang,et al.  Entropy-based subspace clustering for mining numerical data , 1999, KDD '99.

[21]  Pierpaolo D'Urso,et al.  Fuzzy K-means clustering models for triangular fuzzy time trajectories , 2002 .

[22]  H. Kiers.  Towards a standardized notation and terminology in multiway analysis , 2000 .

[23]  B. Frieden,et al.  Physics from Fisher Information: A Unification , 1998 .

[24]  Hidetomo Ichihashi,et al.  Gaussian Mixture PDF Approximation and Fuzzy c-Means Clustering with Entropy Regularization , 2000 .

[25]  Rui-Ping Li,et al.  A maximum-entropy approach to fuzzy clustering , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[26]  G. Battail.  Théorie de l'information , 1982, Électronique.

[27]  P. Groenen,et al.  Data analysis, classification, and related methods , 2000 .

[28]  Deniz Erdoğmuş,et al.  Clustering using Renyi's entropy , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[29]  Geoffrey C. Fox,et al.  A deterministic annealing approach to clustering , 1990, Pattern Recognit. Lett..

[30]  Pierpaolo D'Urso,et al.  Fuzzy Clustering for Data Time Arrays With Inlier and Outlier Time Trajectories , 2005, IEEE Transactions on Fuzzy Systems.

[31]  Michel Ménard,et al.  Extreme physical information and objective function in fuzzy clustering , 2002, Fuzzy Sets Syst..

[32]  José Carlos Príncipe,et al.  Information Theoretic Clustering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Sadaaki Miyamoto,et al.  Fuzzy c-means as a regularization and maximum entropy approach , 1997 .

[34]  E. H. Simpson.  Measurement of Diversity , 1949, Nature.

[35]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[36]  Pierpaolo D'Urso,et al.  Dissimilarity measures for time trajectories , 2000 .

[37]  Masao Mukaidono,et al.  Gaussian clustering method based on maximum-fuzzy-entropy interpretation , 1999, Fuzzy Sets Syst..

[38]  Michel Ménard,et al.  Possibilistic and probabilistic fuzzy clustering: unification within the framework of the non-extensive thermostatistics , 2003, Pattern Recognit..

[39]  Pierpaolo D'Urso.  Fuzzy C-Means Clustering Models For Multivariate Time-Varying Data: Different Approaches , 2004, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[40]  S. Roberts,et al.  Minimum entropy data partitioning , 1999 .