Constrained Clustering Using Column Generation

In recent years, it has been realized that many problems in data mining can be seen as pure optimisation problems. In this work, we investigate the problem of constraint-based clustering from an optimisation point of view. The use of constraints in clustering is a recent development and allows to encode prior beliefs about desirable clusters. This paper proposes a new solution for minimum-sum-of-squares clustering under constraints, where the constraints considered are must-link constraints, cannot-link constraints and anti-monotone constraints on individual clusters. Contrary to most earlier approaches, it is exact and provides a fundamental approach for including these constraints. The proposed approach uses column generation in an integer linear programming setting. The key insight is that these constraints can be pushed into a branch-and-bound algorithm used for generating new columns. Experimental results show the feasibility of the approach and the promise of the branch-and-bound algorithm that solves the subproblem directly.

[1]  Keinosuke Fukunaga,et al.  A Branch and Bound Clustering Algorithm , 1975, IEEE Transactions on Computers.

[2]  I. Stancu-Minasian Nonlinear Fractional Programming , 1997 .

[3]  Stefan Kramer,et al.  Integer Linear Programming Models for Constrained Clustering , 2010, Discovery Science.

[4]  Pierre Hansen,et al.  An improved column generation algorithm for minimum sum-of-squares clustering , 2009, Math. Program..

[5]  Alexander Schrijver,et al.  Combinatorial optimization. Polyhedra and efficiency. , 2003 .

[6]  Jiming Peng,et al.  A Cutting Algorithm for the Minimum Sum-of-Squared Error Clustering , 2005, SDM.

[7]  M. Brusco A Repetitive Branch-and-Bound Procedure for Minimum Within-Cluster Sums of Squares Partitioning , 2006, Psychometrika.

[8]  Pierre Hansen,et al.  An Interior Point Algorithm for Minimum Sum-of-Squares Clustering , 1997, SIAM J. Sci. Comput..

[9]  George L. Nemhauser,et al.  Min-cut clustering , 1993, Math. Program..

[10]  Thomas Hofmann,et al.  Non-redundant data clustering , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[11]  S. S. Ravi,et al.  The complexity of non-hierarchical clustering with instance and cluster level constraints , 2007, Data Mining and Knowledge Discovery.

[12]  G. Diehr Evaluation of a Branch and Bound Algorithm for Clustering , 1985 .

[13]  Jacqueline J. Meulman,et al.  Improving Dynamic Programming Strategies for Partitioning , 2004, J. Classif..

[14]  Claire Cardie,et al.  Clustering with Instance-Level Constraints , 2000, AAAI/IAAI.

[15]  Indrajit Bhattacharya,et al.  Using Assignment Constraints to Avoid Empty Clusters in k-Means Clustering , 2008 .

[16]  F. Sibel Salman,et al.  A mixed-integer programming approach to the clustering problem with an application in customer segmentation , 2006, Eur. J. Oper. Res..

[17]  Tobias Achterberg,et al.  SCIP: solving constraint integer programs , 2009, Math. Program. Comput..

[18]  Peter A. Flach,et al.  Evaluation Measures for Multi-class Subgroup Discovery , 2009, ECML/PKDD.

[19]  Robert E. Jensen,et al.  A Dynamic Programming Algorithm for Cluster Analysis , 1969, Oper. Res..

[20]  Pierre Hansen,et al.  NP-hardness of Euclidean sum-of-squares clustering , 2008, Machine Learning.

[21]  Ian Davidson,et al.  Constrained Clustering: Advances in Algorithms, Theory, and Applications , 2008 .

[22]  P. Hansen,et al.  A branch-and-cut SDP-based algorithm for minimum sum-of-squares clustering , 2008 .

[23]  S. S. Ravi,et al.  A SAT-based Framework for Efficient Constrained Clustering , 2010, SDM.

[24]  Thi-Bich-Hanh Dao,et al.  A Declarative Framework for Constrained Clustering , 2013, ECML/PKDD.