Data Mining by Means of Binary Representation: A Model for Similarity and Clustering

In this paper we outline a new method for clustering that is based on a binary representation of data records. The binary database relates each entity to all possible attribute values (domain) that entity may assume. The resulting binary matrix allows for similarity and clustering calculation by using the positive (‘1’ bits) of the entity vector. We formulate two indexes: Pair Similarity Index (PSI) to measure similarity between two entities and Group Similarity Index (GSI) to measure similarity within a group of entities. A threshold factor for each attribute domain is defined that is dependent on the domain but independent of the number of entities in the group. The similarity measure provides simplicity of storage and efficiency of calculation. A comparison of our similarity index to other indexes is made. Experiments with sample data indicate a 48% improvement of group similarity over standard methods pointing to the potential and merit of the binary approach to clustering and data mining.

[1]  Israel Spiegler,et al.  Storage and retrieval considerations of binary data bases , 1985, Inf. Process. Manag..

[2]  M. Resnik,et al.  Aspects of Scientific Explanation. , 1966 .

[3]  Rolf Stadler,et al.  Discovering Data Mining: From Concept to Implementation , 1997 .

[4]  Israel Spiegler,et al.  Hempel's Raven paradox: a positive approach to cluster analysis , 2000, Comput. Oper. Res..

[5]  Johannes Gehrke,et al.  Mining Very Large Databases , 1999, Computer.

[6]  D. Madigan,et al.  Proceedings : KDD-99 : the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 15-18, 1999, San Diego, California, USA , 1999 .

[7]  Brian Everitt,et al.  Cluster analysis , 1974 .

[8]  Luis G. Vargas,et al.  Choosing Data-Mining Methods for Multiple Classification: Representational and Performance Measurement Implications for Decision Support , 1999, J. Manag. Inf. Syst..

[9]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[10]  Pieter Adriaans,et al.  Data mining , 1996 .

[11]  F. Borgen,et al.  Applying Cluster Analysis in Counseling Psychology Research. , 1987 .

[12]  Ramasamy Uthurusamy,et al.  From Data Mining to Knowledge Discovery: Current Challenges and Future Directions , 1996, Advances in Knowledge Discovery and Data Mining.

[13]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[14]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[15]  David Haussler,et al.  Mining scientific data , 1996, CACM.

[16]  John A. Hartigan,et al.  Clustering Algorithms , 1975 .

[17]  Rakesh Agrawal.  Data mining (Invited talk. Abstract only): crossing the Chasm , 1999, KDD '99.

[18]  Ananth Grama,et al.  Data Mining: From Serendipity to Science - Guest Editors' Introduction , 1999, Computer.

[19]  Paul Gray,et al.  Special Section: Data Mining , 1999, J. Manag. Inf. Syst..