Compact Discrete Representations for Scalable Similarity Search

Compact Discrete Representations for Scalable Similarity Search Mohammad Norouzi Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2016 Scalable similarity search on images, documents, and user activities benefits generic search, data visualization, and recommendation systems. This thesis concerns the design of algorithms and machine learning tools for faster and more accurate similarity search. The proposed techniques advocate the use of discrete codes for representing the similarity structure of data in a compact way. In particular, we will discuss how one can learn to map high-dimensional data onto binary codes with a metric learning approach. Then, we will describe a simple algorithm for fast exact nearest neighbor search in Hamming distance, which exhibits sub-linear query time performance. Going beyond binary codes, we will highlight a compositional generalization of kmeans clustering which maps data points onto integer codes with storage and search costs that grow sub-linearly in the number of cluster centers. This representation improves upon binary codes, and provides an even more precise approximation of Euclidean distance. Experimental results are reported on multiple datasets including a dataset of SIFT descriptors with 1B entries.