Restoration and recognition in a loop

In this paper we present a novel learning based method for restoring and recognizing images of digits that have been blurred using an unknown kernel. The novelty of our work is an iterative loop that alternates between recognition and restoration stages. In the restoration stage we model the image as an undirected graphical model over the image patches with the compatibility functions represented as non-parametric kernel densities. Compatibility functions are initially learned using uniform random samples from the training data. We solve the inference problem by an extended version of the non-parametric belief propagation algorithm in which we introduce the notion of partial messages. We close the loop by using the confidence scores of the recognition to non-uniformly sample from the training set in order to retrain the compatibility functions. We show experimental results on synthetic and license plate images.

[1]  Takeo Kanade,et al.  Limits on super-resolution and how to break them , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[2]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[3]  William T. Freeman,et al.  Learning Low-Level Vision , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[4]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[5]  Owen Carmichael,et al.  Learning Low-level Vision Learning Low-level Vision , 2000 .

[6]  Harry Shum,et al.  A two-step approach to hallucinating faces: global parametric model and local nonparametric model , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[8]  G. B. Smith,et al.  Preface to S. Geman and D. Geman, “Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images” , 1987 .

[9]  Michal Irani,et al.  Improving resolution by image registration , 1991, CVGIP Graph. Model. Image Process..

[10]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  Andrew Blake,et al.  Motion Deblurring and Super-resolution from an Image Sequence , 1996, ECCV.

[12]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[13]  V. Vapnik.  Estimation of Dependences Based on Empirical Data , 2006 .

[14]  Gerhard Lakemeyer,et al.  Exploring artificial intelligence in the new millennium , 2003 .

[15]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[16]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[17]  Andrew Blake,et al.  Super-resolution Enhancement of Video , 2003, AISTATS.

[18]  Andrew Zisserman,et al.  Super-resolution from multiple views using learnt image models , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Michael Isard,et al.  PAMPAS: real-valued graphical models for computer vision , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..