Message passing algorithms for facility location problems

Discrete location analysis is one of the most widely studied branches of operations research, whose applications arise in a wide variety of settings. This thesis describes a powerful new approach to facility location problems - that of message passing inference in probabilistic graphical models. Using this framework, we develop new heuristic algorithms, as well as a new approximation algorithm for a particular problem type. In machine learning applications, facility location can be seen a discrete formulation of clustering and mixture modeling problems. We apply the developed algorithms to such problems in computer vision. We tackle the problem of motion segmentation in video sequences by formulating it as a facility location instance and demonstrate the advantages of message passing algorithms over current segmentation methods.

[1]  J. Stollsteimer A Working Model for Plant Numbers and Locations , 1963 .

[2]  Alfred A. Kuehn,et al.  A Heuristic Program for Locating Warehouses , 1963 .

[3]  A. Manne Plant Location Under Economies-of-Scale---Decentralization and Computation , 1964 .

[4]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[5]  J. Krarup,et al.  Sharp Lower Bounds and Efficient Algorithms for the Simple Plant Location Problem , 1977 .

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[8]  Dorit S. Hochbaum,et al.  Heuristics for the fixed cost median problem , 1982, Math. Program..

[9]  Judea Pearl,et al.  Reverend Bayes on Inference Engines: A Distributed Hierarchical Approach , 1982, AAAI.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  R. D. Galvão,et al.  A method for solving to optimality uncapacitated location problems , 1990 .

[12]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Bhaba R. Sarker,et al.  Discrete location theory , 1991 .

[14]  Said Salhi,et al.  Discrete Location Theory , 1991 .

[15]  Carsten Lund,et al.  On the hardness of approximating minimization problems , 1993, STOC.

[16]  J. Beasley Lagrangean heuristics for location problems , 1993 .

[17]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[18]  Dariush Divsalar,et al.  Soft-Output Decoding Algorithms in Iterative Decoding of Turbo Codes , 1996 .

[19]  Dorit S. Hochbaum,et al.  Approximation Algorithms for NP-Hard Problems , 1996 .

[20]  Éva Tardos,et al.  Approximation algorithms for facility location problems (extended abstract) , 1997, STOC '97.

[21]  Dorit S. Hochba,et al.  Approximation Algorithms for NP-Hard Problems , 1997, SIGA.

[22]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Jun Zhang,et al.  Pace recognition: eigenface, elastic matching, and neural nets , 1997, Proc. IEEE.

[24]  U. Feige A threshold of ln n for approximating set cover , 1998, JACM.

[25]  Rajmohan Rajaraman,et al.  Analysis of a local search heuristic for facility location problems , 2000, SODA '98.

[26]  P. Torr Geometric motion segmentation and model selection , 1998, Philosophical Transactions of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[27]  Samir Khuller,et al.  Greedy strikes back: improved facility location algorithms , 1998, SODA '98.

[28]  Michael I. Jordan,et al.  Loopy Belief Propagation for Approximate Inference: An Empirical Study , 1999, UAI.

[29]  Christopher M. Bishop,et al.  Mixtures of Probabilistic Principal Component Analyzers , 1999, Neural Computation.

[30]  Mohammad Abdulrahman Al-Fawzan,et al.  A tabu search approach to the uncapacitated facility location problem , 1999, Ann. Oper. Res..

[31]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[32]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[33]  Jozef Kratica,et al.  Solving the simple plant location problem by genetic algorithm , 2001, RAIRO Oper. Res..

[34]  Vijay V. Vazirani,et al.  Approximation algorithms for metric facility location and k-Median problems using the primal-dual schema and Lagrangian relaxation , 2001, JACM.

[35]  Kamesh Munagala,et al.  Web caching using access statistics , 2001, SODA '01.

[36]  Rajmohan Rajaraman,et al.  Approximation algorithms for data placement in arbitrary networks , 2001, SODA '01.

[37]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[38]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[39]  Kenichi Kanatani,et al.  Motion segmentation by subspace separation and model selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[40]  Zvi Drezner,et al.  Facility location - applications and theory , 2001 .

[41]  Maxim Sviridenko An Improved Approximation Algorithm for the Metric Uncapacitated Facility Location Problem , 2002, IPCO.

[42]  Mohammad Mahdian,et al.  Improved Approximation Algorithms for Metric Facility Location Problems , 2002, APPROX.

[43]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[44]  Amin Saberi,et al.  A new greedy approach for facility location problems , 2002, STOC '02.

[45]  Vijay V. Vazirani,et al.  Approximation Algorithms , 2001, Springer Berlin Heidelberg.

[46]  Hongyuan Zha,et al.  Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment , 2002, ArXiv.

[47]  David J. Kriegman,et al.  Clustering appearances of objects under varying illumination conditions , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[48]  Fabián A. Chudak,et al.  Improved Approximation Algorithms for the Uncapacitated Facility Location Problem , 2003, SIAM J. Comput..

[49]  Martin J. Wainwright,et al.  Tree-based reparameterization framework for analysis of sum-product and related algorithms , 2003, IEEE Trans. Inf. Theory.

[50]  Kamesh Munagala,et al.  Local Search Heuristics for k-Median and Facility Location Problems , 2004, SIAM J. Comput..

[51]  Pascal Van Hentenryck,et al.  A simple tabu search for warehouse location , 2004, Eur. J. Oper. Res..

[52]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[53]  Kenichi Kanatani,et al.  Geometric Structure of Degeneracy for Multi-body Motion Segmentation , 2004, ECCV Workshop SMVP.

[54]  David A. Forsyth,et al.  The Joy of Sampling , 2004, International Journal of Computer Vision.

[55]  Jiri Matas,et al.  Two-view geometry estimation unaffected by a dominant plane , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[56]  Yuri Kochetov,et al.  Computationally Difficult Instances for the Uncapacitated Facility Location Problem , 2005 .

[57]  Minghe Sun,et al.  A Tabu Search Heuristic for the Uncapacitated Facility Location Problem , 2005 .

[58]  Sudipto Guha,et al.  Improved Combinatorial Algorithms for Facility Location Problems , 2005, SIAM J. Comput..

[59]  S. Shankar Sastry,et al.  Generalized principal component analysis (GPCA) , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Mohammad Mahdian,et al.  Approximation Algorithms for Metric Facility Location Problems , 2006, SIAM J. Comput..

[61]  David J. Fleet,et al.  Temporal motion models for monocular and multiview 3D human body tracking , 2006, Comput. Vis. Image Underst..

[62]  Marc Pollefeys,et al.  A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-rigid, Degenerate and Non-degenerate , 2006, ECCV.

[63]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[64]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[65]  David Suter,et al.  Two-View Multibody Structure-and-Motion with Outliers through Model Selection , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[66]  Allen Y. Yang,et al.  Robust Statistical Estimation and Segmentation of Multiple Subspaces , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[67]  Hongdong Li,et al.  Two-View Motion Segmentation from Linear Programming Relaxation , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[68]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[70]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[71]  Kay Römer,et al.  Distributed Facility Location Algorithms for Flexible Configuration of Wireless Sensor Networks , 2007, DCOSS.

[72]  M. Charikar,et al.  An Optimal Bifactor Approximation Algorithm for the Metric Uncapacitated Facility Location Problem , 2006, SIAM J. Comput..

[73]  Delbert Dueck,et al.  Clustering by Passing Messages Between Data Points , 2007, Science.

[74]  Rahul Gupta,et al.  Efficient inference with cardinality-based clique potentials , 2007, ICML '07.

[75]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[76]  Polina Golland,et al.  Convex Clustering with Exemplar-Based Models , 2007, NIPS.

[77]  João Paulo Costeira,et al.  Subspace segmentation with outliers: A grassmannian approach to the maximum consensus subspace , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[78]  Nikos Komodakis,et al.  Beyond Loose LP-Relaxations: Optimizing MRFs by Repairing Cycles , 2008, ECCV.

[79]  Brendan J. Frey,et al.  Constructing Treatment Portfolios Using Affinity Propagation , 2008, RECOMB.

[80]  Brendan J. Frey,et al.  Flexible Priors for Exemplar-based Clustering , 2008, UAI.

[81]  R. Darlington,et al.  Factor Analysis , 2008 .

[82]  Brendan J. Frey,et al.  FLoSS: Facility location for subspace segmentation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[83]  Brendan J. Frey,et al.  A Binary Variable Model for Affinity Propagation , 2009, Neural Computation.

[84]  Delbert Dueck,et al.  Affinity Propagation: Clustering Data by Passing Messages , 2009 .

[85]  Tommi S. Jaakkola,et al.  Tree Block Coordinate Descent for MAP in Graphical Models , 2009, AISTATS.

[86]  Richard S. Zemel,et al.  HOP-MAP: Efficient Message Passing with High Order Potentials , 2010, AISTATS.

[87]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.