Distributed Discrete Hashing by Equivalent Continuous Formulation

Hashing based approximate nearest neighbor search has attracted considerable attention in various fields. Most of the existing hashing methods are centralized, which cannot be used for many large-scale applications with the data stored or collected in a distributed manner. In this article, we consider the distributed hashing problem. The main difficulty of hashing is brought by its inherent binary constraints, which makes the problem generally NP-hard. Most of the existing distributed hashing methods chose to relax the problem by dropping the binary constraints. However, such a manner will bring additional quantization error, which makes the binary codes less effective. In this paper, we propose a novel distributed discrete hashing method, which learns effective hash codes without using any relaxations. Specifically, we give a method to transform the discrete hashing problem into an equivalent distributed continuous optimization problem. After transformation, we devise a distributed discrete hashing (dDH) algorithm based on the idea of DC programming to solve the problem. To obtain more efficient hash codes, we further add bits balance and uncorrelation constraints to the hashing problem, and we also propose a distributed constrained discrete hashing algorithm (dCDH) to solve this problem. Extensive experiments are provided to show the superiority of the proposed methods.

[1]  Cheng Deng,et al.  Unsupervised Deep Generative Adversarial Hashing Network , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[3]  Bharadwaj Veeravalli,et al.  Distributed Image Processing On A Network Of Workstations , 2003 .

[4]  Xiangfeng Wang,et al.  Multi-Agent Distributed Optimization via Inexact Consensus ADMM , 2014, IEEE Transactions on Signal Processing.

[5]  Wen Gao,et al.  Supervised Distributed Hashing for Large-Scale Multimedia Retrieval , 2018, IEEE Transactions on Multimedia.

[6]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[7]  David J. Fleet,et al.  Minimal Loss Hashing for Compact Binary Codes , 2011, ICML.

[8]  Alexandr Andoni,et al.  Near-Optimal Hashing Algorithms for Approximate Nearest Neighbor in High Dimensions , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[9]  Don Coppersmith,et al.  On the Asymptotic Complexity of Matrix Multiplication , 1982, SIAM J. Comput..

[10]  Wei Liu,et al.  Hashing with Graphs , 2011, ICML.

[11]  Asuman E. Ozdaglar,et al.  Distributed Subgradient Methods for Multi-Agent Optimization , 2009, IEEE Transactions on Automatic Control.

[12]  Svetha Venkatesh,et al.  Distributed query processing for mobile surveillance , 2007, ACM Multimedia.

[13]  Stephen P. Boyd,et al.  Fast linear iterations for distributed averaging , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[14]  Wei Liu,et al.  Learning to Hash for Indexing Big Data—A Survey , 2015, Proceedings of the IEEE.

[15]  Le Thi Hoai An,et al.  The DC (Difference of Convex Functions) Programming and DCA Revisited with DC Models of Real World Nonconvex Optimization Problems , 2005, Ann. Oper. Res..

[16]  Rongrong Ji,et al.  Supervised hashing with kernels , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Guosheng Lin,et al.  Learning Hash Functions Using Column Generation , 2013, ICML.

[18]  Michael W. Berry,et al.  Large-Scale Information Retrieval with Latent Semantic Indexing , 1997, Inf. Sci..

[19]  Gaurav S. Sukhatme,et al.  Mobile Sensor Network Deployment using Potential Fields : A Distributed , Scalable Solution to the Area Coverage Problem , 2002 .

[20]  Chunguang Li,et al.  Distributed Robust Optimization in Networked System , 2017, IEEE Transactions on Cybernetics.

[21]  Zhi-Hua Zhou,et al.  Column Sampling Based Discrete Supervised Hashing , 2016, AAAI.

[22]  Xianglong Liu,et al.  DisITQ: A Distributed Iterative Quantization Hashing Learning Algorithm , 2016, 2016 9th International Symposium on Computational Intelligence and Design (ISCID).

[23]  Shih-Fu Chang,et al.  Semi-Supervised Hashing for Large-Scale Search , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Stephen P. Boyd,et al.  Subgradient Methods , 2007 .

[25]  Prateek Jain,et al.  Fast Similarity Search for Learned Metrics , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Le Thi Hoai An,et al.  Large-Scale Molecular Optimization from Distance Matrices by a D.C. Optimization Approach , 2003, SIAM J. Optim..

[27]  Xianglong Liu,et al.  Structure Sensitive Hashing With Adaptive Product Quantization , 2016, IEEE Transactions on Cybernetics.

[28]  Tieniu Tan,et al.  Supervised Discrete Hashing With Relaxation , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Svetlana Lazebnik,et al.  Iterative quantization: A procrustean approach to learning binary codes , 2011, CVPR 2011.

[30]  Huchuan Lu,et al.  Sparse Hashing Tracking , 2016, IEEE Transactions on Image Processing.

[31]  Piotr Indyk,et al.  Similarity Search in High Dimensions via Hashing , 1999, VLDB.

[32]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[33]  Antonio Torralba,et al.  Spectral Hashing , 2008, NIPS.

[34]  Xuelong Li,et al.  Spectral Embedded Hashing for Scalable Image Retrieval , 2014, IEEE Transactions on Cybernetics.

[35]  Christopher Frost,et al.  Spanner: Google's Globally-Distributed Database , 2012, OSDI.

[36]  Hui-Liang Shen,et al.  Equivalent Continuous Formulation of General Hashing Problem , 2019, IEEE Transactions on Cybernetics.

[37]  Wei Liu,et al.  Supervised Discrete Hashing , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Miguel Á. Carreira-Perpiñán,et al.  Hashing with binary autoencoders , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Wei Liu,et al.  Discrete Graph Hashing , 2014, NIPS.

[40]  Rongrong Ji,et al.  Ordinal Constrained Binary Code Learning for Nearest Neighbor Search , 2016, AAAI.

[41]  Yang Yang,et al.  A Fast Optimization Method for General Binary Code Learning , 2016, IEEE Transactions on Image Processing.

[42]  Hui-Liang Shen,et al.  Distributed Graph Hashing , 2019, IEEE Transactions on Cybernetics.

[43]  Ngai-Man Cheung,et al.  Learning to Hash with Binary Deep Neural Network , 2016, ECCV.

[44]  Xianglong Liu,et al.  Distributed Adaptive Binary Quantization for Fast Nearest Neighbor Search , 2017, IEEE Transactions on Image Processing.

[45]  Junjie Wu,et al.  Adaptive Binary Quantization for Fast Nearest Neighbor Search , 2016, ECAI.

[46]  Wu-Jun Li,et al.  Scalable Graph Hashing with Feature Transformation , 2015, IJCAI.

[47]  Jiwen Lu,et al.  Cost-Sensitive Local Binary Feature Learning for Facial Age Estimation , 2015, IEEE Transactions on Image Processing.

[48]  Nicole Immorlica,et al.  Locality-sensitive hashing scheme based on p-stable distributions , 2004, SCG '04.

[49]  Junli Liang,et al.  Distributed Dictionary Learning for Sparse Representation in Sensor Networks , 2014, IEEE Transactions on Image Processing.

[50]  Xi Zhang,et al.  Hashing for Distributed Data , 2015, ICML.

[51]  Minyi Guo,et al.  Supervised hashing with latent factor models , 2014, SIGIR.