Model Selection and Estimation of a Finite Shifted-Scaled Dirichlet Mixture Model

This paper proposes an unsupervised learning algorithm for a finite mixture model of shifted-scaled Dirichlet distributions. Maximum likelihood and Newton raphson approaches are used for parameters estimation. In this research work, we address the flexibility challenge of the Dirichlet distribution by having another set of parameters for the location (beside the Scale parameter) that add functional probability models. This paper evaluates the capability of the discussed model to perform the categorization using both synthetic and real data related to the medical science to help in selecting wart treatment method, in the business field to detect the reasons behind employees' absenteeism, and the writer identification application to define the author of off-line handwritten documents. We also compare the model performance against scaled Dirichlet, the classic Dirichlet, and Gaussian mixture models. Finally, experimental results are presented on the selected datasets. Besides, we apply the minimum message length to determine the optimal number of the components found within each dataset.

[1]  Anirban Mahanti,et al.  Traffic classification using clustering algorithms , 2006, MineNet '06.

[2]  Jonathan Huang Maximum Likelihood Estimation of Dirichlet Distribution Parameters , 2005 .

[3]  Yannis Manolopoulos,et al.  Data Mining techniques for the detection of fraudulent financial statements , 2007, Expert Syst. Appl..

[4]  Lambert Schomaker,et al.  Junction detection in handwritten documents and its application to writer identification , 2015, Pattern Recognit..

[5]  Louis Vuurpijl,et al.  Writer identification using edge-based directional features , 2003, Seventh International Conference on Document Analysis and Recognition, 2003. Proceedings..

[6]  Saeid Nahavandi,et al.  An expert system for selecting wart treatment method , 2017, Comput. Biol. Medicine.

[7]  T. Minka Estimating a Dirichlet distribution , 2012 .

[8]  Nizar Bouguila,et al.  Using unsupervised learning of a finite Dirichlet mixture model to improve pattern recognition applications , 2005, Pattern Recognit. Lett..

[9]  Nizar Bouguila,et al.  Predicting Defect-Prone Software Modules Using Shifted-Scaled Dirichlet Distribution , 2018, 2018 First International Conference on Artificial Intelligence for Industries (AI4I).

[10]  V. Pawlowsky-Glahn,et al.  Simplicial geometry for compositional data , 2006, Geological Society, London, Special Publications.

[11]  Horst Bunke,et al.  The IAM-database: an English sentence database for offline handwriting recognition , 2002, International Journal on Document Analysis and Recognition.

[12]  M. Pechwitz,et al.  IFN/ENIT: database of handwritten arabic words , 2002 .

[13]  Andreas K. Maier,et al.  Unsupervised Feature Learning for Writer Identification and Writer Retrieval , 2017, 2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR).

[14]  Nizar Bouguila,et al.  On Fitting Finite Dirichlet Mixture Using ECM and MML , 2005, ICAPR.

[15]  James O. Berger,et al.  Ockham's Razor and Bayesian Analysis , 1992 .

[16]  Nizar Bouguila,et al.  A finite mixture model for simultaneous high-dimensional clustering, localized feature selection and outlier rejection , 2012, Expert Syst. Appl..

[17]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[18]  Youbao Tang,et al.  Offline Text-Independent Writer Identification Based on Scale Invariant Feature Transform , 2014, IEEE Transactions on Information Forensics and Security.

[19]  Zhengyu Hu,et al.  Initializing the EM Algorithm for Data Clustering and Sub-population Detection , 2015 .

[20]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[21]  H. Akaike A new look at the statistical model identification , 1974 .

[22]  David L. Dowe,et al.  MML clustering of multi-state, Poisson, von Mises circular and Gaussian distributions , 2000, Stat. Comput..

[23]  C. S. Wallace,et al.  Estimation and Inference by Compact Coding , 1987 .

[24]  P. Powell Calculating Determinants of Block Matrices , 2011, 1112.4379.

[25]  Nizar Bouguila,et al.  A Model-Based Approach for Discrete Data Clustering and Feature Weighting Using MAP and Stochastic Complexity , 2009, IEEE Transactions on Knowledge and Data Engineering.

[26]  G. Ronning Maximum likelihood estimation of dirichlet distributions , 1989 .

[27]  Nizar Bouguila,et al.  Unsupervised selection of a finite Dirichlet mixture model: an MML-based approach , 2006, IEEE Transactions on Knowledge and Data Engineering.

[28]  Nizar Bouguila,et al.  A countably infinite mixture model for clustering and feature selection , 2011, Knowledge and Information Systems.

[29]  Nizar Bouguila,et al.  Count Data Modeling and Classification Using Finite Mixtures of Distributions , 2011, IEEE Transactions on Neural Networks.

[30]  Nizar Bouguila,et al.  Variational learning of a Dirichlet process of generalized Dirichlet distributions for simultaneous clustering and feature selection , 2013, Pattern Recognit..

[31]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[32]  Mohammad Alshayeb,et al.  KHATT: Arabic Offline Handwritten Text Database , 2012, 2012 International Conference on Frontiers in Handwriting Recognition.

[33]  Nizar Bouguila,et al.  High-Dimensional Unsupervised Selection and Estimation of a Finite Generalized Dirichlet Mixture Model Based on Minimum Message Length , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Anil K. Jain,et al.  Unsupervised Learning of Finite Mixture Models , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Nizar Bouguila,et al.  Positive vectors clustering using inverted Dirichlet finite mixture models , 2012, Expert Syst. Appl..

[36]  Nizar Bouguila,et al.  Finite asymmetric generalized Gaussian mixture models learning for infrared object detection , 2013, Comput. Vis. Image Underst..

[37]  Nizar Bouguila,et al.  Bayesian hybrid generative discriminative learning based on finite Liouville mixture models , 2011, Pattern Recognit..

[38]  R. Alizadehsani,et al.  Intralesional immunotherapy compared to cryotherapy in the treatment of warts , 2017, International journal of dermatology.

[39]  Jonathan J. Oliver,et al.  Finding overlapping components with MML , 2000, Stat. Comput..

[40]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .