Distributed Algorithm for Computing Formal Concepts Using Map-Reduce Framework

Searching for interesting patterns in binary matrices plays an important role in data mining and, in particular, in formal concept analysis and related disciplines. Several algorithms for computing particular patterns represented by maximal rectangles in binary matrices were proposed but their major drawback is their computational complexity limiting their application on relatively small datasets. In this paper we introduce a scalable distributed algorithm for computing maximal rectangles that uses the map-reduce approach to data processing.

[1]  Bernhard Ganter,et al.  Formal Concept Analysis: Mathematical Foundations , 1998 .

[2]  Vilém Vychodil,et al.  Parallel algorithm for computing fixpoints of Galois connections , 2010, Annals of Mathematics and Artificial Intelligence.

[3]  Bernhard Ganter,et al.  Two Basic Algorithms in Concept Analysis , 2010, ICFCA.

[4]  Rudolf Wille,et al.  Restructuring Lattice Theory: An Approach Based on Hierarchies of Concepts , 2009, ICFCA.

[5]  Sergei O. Kuznetsov,et al.  Learning of Simple Conceptual Graphs from Positive and Negative Examples , 1999, PKDD.

[6]  Christian Lindig.  Fast Concept Analysis , 2000 .

[7]  Peter Øhrstrøm,et al.  Working with Conceptual Structures - Contributions to ICCS 2000 , 2000 .

[8]  Bernhard Ganter,et al.  Formal Concept Analysis, 6th International Conference, ICFCA 2008, Montreal, Canada, February 25-28, 2008, Proceedings , 2008, International Conference on Formal Concept Analysis.

[9]  Johannes Fürnkranz,et al.  Knowledge Discovery in Databases: PKDD 2006, 10th European Conference on Principles and Practice of Knowledge Discovery in Databases, Berlin, Germany, September 18-22, 2006, Proceedings , 2006, PKDD.

[10]  Sergei O. Kuznetsov,et al.  Comparing performance of algorithms for generating concept lattices , 2002, J. Exp. Theor. Artif. Intell..

[11]  Vilém Vychodil,et al.  Discovery of optimal factors in binary data via a novel method of matrix decomposition , 2010, J. Comput. Syst. Sci..

[12]  Claudio Carpineto,et al.  Concept data analysis - theory and applications , 2004 .

[13]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[14]  Engelbert Mephu Nguifo,et al.  A Parallel Algorithm to Generate Formal Concepts for Large Data , 2004, ICFCA.

[15]  Anne Berry,et al.  A local approach to concept generation , 2007, Annals of Mathematics and Artificial Intelligence.

[16]  Petko Valtchev,et al.  A Parallel Algorithm for Lattice Construction , 2005, ICFCA.

[17]  Jan Outrata,et al.  Parallel Recursive Algorithm for FCA , 2008 .

[18]  Jan Komorowski,et al.  Principles of Data Mining and Knowledge Discovery , 2001, Lecture Notes in Computer Science.

[19]  Pauli Miettinen,et al.  The Discrete Basis Problem , 2006, IEEE Transactions on Knowledge and Data Engineering.