Effective reinforcement learning based local search for the maximum k-plex problem

The maximum k-plex problem is a computationally complex problem, which emerged from graph-theoretic social network studies. This paper presents an effective hybrid local search for solving the maximum k-plex problem that combines the recently proposed breakout local search algorithm with a reinforcement learning strategy. The proposed approach includes distinguishing features such as: a unified neighborhood search based on the swapping operator, a distance-and-quality reward for actions and a new parameter control mechanism based on reinforcement learning. Extensive experiments for the maximum k-plex problem (k = 2, 3, 4, 5) on 80 benchmark instances from the second DIMACS Challenge demonstrate that the proposed approach can match the best-known results from the literature in all but four problem instances. In addition, the proposed algorithm is able to find 32 new best solutions.

[1]  Sergiy Butenko,et al.  Clique Relaxations in Social Network Analysis: The Maximum k-Plex Problem , 2011, Oper. Res..

[2]  Mark Hoogendoorn,et al.  Why parameter control mechanisms should be benchmarked against random variation , 2013, 2013 IEEE Congress on Evolutionary Computation.

[3]  Ravi Kumar,et al.  Discovering Large Dense Subgraphs in Massive Graphs , 2005, VLDB.

[4]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[5]  Sergiy Butenko,et al.  A network-based data mining approach to portfolio selection via weighted clique relaxations , 2014, Ann. Oper. Res..

[6]  Illya V. Hicks,et al.  Combinatorial algorithms for the maximum k-plex problem , 2012, J. Comb. Optim..

[7]  Qinghua Wu,et al.  A review on algorithms for maximum clique problems , 2015, Eur. J. Oper. Res..

[8]  M. Trick,et al.  Cliques, Coloring, and Satisfiability: Second DIMACS Implementation Challenge, Workshop, October 11-13, 1993 , 1996 .

[9]  Bin Wu,et al.  Community detection in large-scale social networks , 2007, WebKDD/SNA-KDD '07.

[10]  Celso C. Ribeiro,et al.  Greedy Randomized Adaptive Search Procedures: Advances, Hybridizations, and Applications , 2010 .

[11]  Leslie Pérez Cáceres,et al.  The irace package: Iterated racing for automatic algorithm configuration , 2016 .

[12]  Jin-Kao Hao,et al.  Breakout Local Search for maximum clique problems , 2013, Comput. Oper. Res..

[13]  Stephen B. Seidman,et al.  A graph‐theoretic generalization of the clique concept* , 1978 .

[14]  Holger H. Hoos,et al.  Automated Algorithm Configuration and Parameter Tuning , 2012, Autonomous Search.

[15]  David S. Johnson,et al.  Cliques, Coloring, and Satisfiability , 1996 .

[16]  Sergiy Butenko,et al.  Algorithms for detecting optimal hereditary structures in graphs, with application to clique relaxations , 2013, Computational Optimization and Applications.

[17]  Amirhossein Meisami,et al.  A Hybrid Metaheuristic for the Maximum k-Plex Problem , 2014, Examining Robustness and Vulnerability of Networked Systems.

[18]  Jin-Kao Hao,et al.  Reinforcement learning based local search for grouping problems: A case study on graph coloring , 2016, Expert Syst. Appl..

[19]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[20]  Valdis E. Krebs,et al.  Mapping Networks of Terrorist Cells , 2001 .

[21]  Yi Zhou,et al.  Frequency-driven tabu search for the maximum s-plex problem , 2017, Comput. Oper. Res..

[22]  Edmund K. Burke,et al.  A hybrid breakout local search and reinforcement learning approach to the vertex separator problem , 2017, Eur. J. Oper. Res..

[23]  M. Newman,et al.  The structure of scientific collaboration networks. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Mingyu Xiao,et al.  On a generalization of Nemhauser and Trotter's local optimization theorem , 2016, J. Comput. Syst. Sci..

[25]  Rolf Niedermeier,et al.  Exact combinatorial algorithms and experiments for finding maximum k-plexes , 2012, J. Comb. Optim..

[26]  Jin-Kao Hao,et al.  Breakout Local Search for the Max-Cutproblem , 2013, Eng. Appl. Artif. Intell..