Algorithmic Differentiation: Application to Variational Problems in Computer Vision

Many vision problems can be formulated as minimization of appropriate energy functionals. These energy functionals are usually minimized, based on the calculus of variations (Euler-Lagrange equation). Once the Euler-Lagrange equation has been determined, it needs to be discretized in order to implement it on a digital computer. This is not a trivial task and, is moreover, error- prone. In this paper, we propose a flexible alternative. We discretize the energy functional and, subsequently, apply the mathematical concept of algorithmic differentiation to directly derive algorithms that implement the energy functional's derivatives. This approach has several advantages: First, the computed derivatives are exact with respect to the implementation of the energy functional. Second, it is basically straightforward to compute second-order derivatives and, thus, the Hessian matrix of the energy functional. Third, algorithmic differentiation is a process which can be automated. We demonstrate this novel approach on three representative vision problems (namely, denoising, segmentation, and stereo) and show that state-of-the-art results are obtained with little effort.

[1]  Michael Hintermüller,et al.  An Inexact Newton-CG-Type Active Contour Approach for the Minimization of the Mumford-Shah Functional , 2004, Journal of Mathematical Imaging and Vision.

[2]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[3]  J. Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[4]  Rachid Deriche,et al.  Regularization, Scale-Space, and Edge Detection Filters , 1996, Journal of Mathematical Imaging and Vision.

[5]  A. Griewank,et al.  Automatic differentiation of algorithms : theory, implementation, and application , 1994 .

[6]  直樹 武川,et al.  Regularization , 2019, Encyclopedia of Continuum Mechanics.

[7]  Bernhard Kawohl,et al.  From Mumford–Shah to Perona–Malik in image processing , 2004 .

[8]  R. Stephenson A and V , 1962, The British journal of ophthalmology.

[9]  Christian Bischof,et al.  Automatic differentiation, tangent linear models, and (pseudo) adjoints , 1995 .

[10]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[11]  Ron Kimmel,et al.  Variational Restoration and Edge Detection for Color Images , 2003, Journal of Mathematical Imaging and Vision.

[12]  Griewank,et al.  On automatic differentiation , 1988 .

[13]  Tony F. Chan,et al.  Total variation blind deconvolution , 1998, IEEE Trans. Image Process..

[14]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[15]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[16]  Riccardo March,et al.  A variational method for the recovery of smooth boundaries , 1997, Image Vis. Comput..

[17]  Shen,et al.  Piecewise H − 1 + H 0 + H 1 Images and the Mumford-Shah-Sobolev Model for Segmented Image Decomposition Jianhong ( , 2005 .

[18]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[19]  Tony F. Chan,et al.  Active contours without edges , 2001, IEEE Trans. Image Process..

[20]  Laurent D. Cohen,et al.  Image Registration, Optical Flow and Local Rigidity , 2001, Journal of Mathematical Imaging and Vision.

[21]  M. Nikolova An Algorithm for Total Variation Minimization and Applications , 2004 .

[22]  D. Mumford,et al.  Optimal approximations by piecewise smooth functions and associated variational problems , 1989 .

[23]  Mario Bertero,et al.  The Stability of Inverse Problems , 1980 .

[24]  Jorge Nocedal,et al.  Large Scale Unconstrained Optimization , 1997 .

[25]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[26]  Thomas Brox,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Highly Accurate Optic Flow Computation with Theoretically Justified Warping Highly Accurate Optic Flow Computation with Theoretically Justified Warping , 2022 .

[27]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[28]  Josiane Zerubia,et al.  A Variational Model for Image Classification and Restoration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Luminita A. Vese,et al.  Multiphase Object Detection and Image Segmentation , 2003 .

[30]  Bart M. ter Haar Romeny,et al.  Geometry-Driven Diffusion in Computer Vision , 1994, Computational Imaging and Vision.

[31]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[32]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[33]  Jean-Michel Morel,et al.  Variational methods in image segmentation , 1995 .

[34]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  J. Hadamard Sur les problemes aux derive espartielles et leur signification physique , 1902 .

[36]  A. Chambolle Practical, Unified, Motion and Missing Data Treatment in Degraded Video , 2004, Journal of Mathematical Imaging and Vision.

[37]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[38]  Rachid Deriche,et al.  Regularization, Scale-Space, and Edge Detection Filters , 1996, ECCV.

[39]  Thomas Slawig,et al.  Generating efficient derivative code with TAF: Adjoint and tangent linear Euler flow around an airfoil , 2005, Future Gener. Comput. Syst..

[40]  Michel Barlaud,et al.  Variational approach for edge-preserving regularization using coupled PDEs , 1998, IEEE Trans. Image Process..

[41]  Rachid Deriche,et al.  Computing Optical Flow via Variational Techniques , 1999, SIAM J. Appl. Math..

[42]  L. Ambrosio,et al.  Approximation of functional depending on jumps by elliptic functional via t-convergence , 1990 .

[43]  Andy M. Yip,et al.  Recent Developments in Total Variation Image Restoration , 2004 .

[44]  Thomas Kaminski,et al.  Automatic Sparsity Detection Implemented as a Source-to-Source Transformation , 2006, International Conference on Computational Science.

[45]  Jianhong Shen,et al.  Gamma-Convergence Approximation to Piecewise Constant Mumford-Shah Segmentation , 2005, ACIVS.

[46]  X. Zou,et al.  Introduction to Adjoint Techniques and the MM5 Adjoint Modeling System. , 1997 .

[47]  Josien P. W. Pluim,et al.  Image registration , 2003, IEEE Transactions on Medical Imaging.

[48]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[49]  Alper Yilmaz,et al.  Level Set Methods , 2007, Wiley Encyclopedia of Computer Science and Engineering.

[50]  Thomas Kaminski,et al.  Recipes for adjoint code construction , 1998, TOMS.

[51]  Jianhong Shen,et al.  Digital inpainting based on the Mumford–Shah–Euler image model , 2002, European Journal of Applied Mathematics.

[52]  A. Chambolle,et al.  Inverse problems in image processing and image segmentation : some mathematical and numerical aspects , 2000 .

[53]  Antonin Chambolle,et al.  Total Variation Minimization and a Class of Binary MRF Models , 2005, EMMCVPR.

[54]  Andrew Blake,et al.  Visual Reconstruction , 1987, Deep Learning for EEG-Based Brain–Computer Interfaces.

[55]  D. Mumford The Bayesian Rationale for Energy Functionals 1 , 1994 .

[56]  Der Fakult,et al.  On Variational Problems and Gradient Flows in Image Processing , 2005 .

[57]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[58]  X. Yi On Automatic Differentiation , 2005 .

[59]  Donald Geman,et al.  Nonlinear image recovery with half-quadratic regularization , 1995, IEEE Trans. Image Process..

[60]  Felice Andrea Pellegrino,et al.  Self-adaptive regularization , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Jianhong Shen,et al.  Piecewise H−1+H0+H1 images and the Mumford-Shah-Sobolevmodel for segmented image decomposition , 2005 .

[62]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[63]  M. J. D. Powell,et al.  on The state of the art in numerical analysis , 1987 .

[64]  B. Vemuri,et al.  A level-set based approach to image registration , 2000, Proceedings IEEE Workshop on Mathematical Methods in Biomedical Image Analysis. MMBIA-2000 (Cat. No.PR00737).

[65]  Tony F. Chan,et al.  Euler's Elastica and Curvature-Based Inpainting , 2003, SIAM J. Appl. Math..

[66]  M. Bertero,et al.  Ill-posed problems in early vision , 1988, Proc. IEEE.

[67]  Andreas Griewank,et al.  The chain rule revisited in scientific computing. , 1991 .

[68]  Yair Shapira,et al.  Solving PDEs in C++: Numerical Methods in a Unified Object-Oriented Approach , 2006 .

[69]  Hans-Hellmut Nagel,et al.  An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.