HADES: Hierarchical Approximate Decoding for Structured Prediction

Many fundamental tasks in machine learning require predicting complex objects rather than a simple yes-no answer or a number. Structured Output Prediction deals with learning such complex objects, which model an inherent structure between interdependent variables. Recently, large margin methods like Structured SVMs (SSVM) have gained popularity to solve this task due to efficient and generalized optimization techniques. These optimization algorithms typically rely on solving an inference or decoding sub-problem every iteration, which is computationally expensive. Moreover, little is known on learning a structured model by solving this sub-problem approximately. To address these issues, this thesis introduces a generalized technique to learn from a series of coarse-to-fine approximate candidates based on the recent Block-Coordinate Frank-Wolfe algorithm for Structured SVMs. The core observation we use is that one can learn to a reasonable degree, even from approximate solutions. This technique is presented in the context of a popular Computer Vision problem – Semantic Image Segmentation. We pose the decoding sub-problem to that of solving a series of increasingly complex surrogate Conditional Random Fields (CRF) in search for a candidate which meets the required approximation quality. We evaluate our technique in the context of natural scene image segmentation on the MSRC-21 dataset. Our experiments indicate that even extremely approximate solutions, which are 50x faster to decode, contribute to learning using our strategy. We achieve the same accuracy as our baseline and in addition, we reach a reasonable accuracy 1.5x-4x as quickly.