A usability case study of algorithmic differentiation tools on the ISSM ice sheet model

Algorithmic differentiation (AD) based on operator overloading is often the only feasible approach for applying AD in complex C++ software environments. Challenges pertaining to the introduction of an AD tool based on operator overloading have been studied in the past. However, in order to assess possible performance gains or to verify derivative values, it is advantageous to be able to apply more than one AD tool to a given code. Hence, in this work, we investigate usability issues when exchanging AD tools. Our study is based on the NASA/JPL/UCI Ice Sheet System Model (ISSM) which currently employs the AD tool ADOL-C. We introduce CoDiPack to ISSM, a more recent AD tool offering a similar set of features while promising performance improvements. In addition to the obvious type change for the AD-augmented float type, this transition requires the change to a different adjoint MPI library, adaptation of the MUMPS solver wrapper, and changes to the derivative seeding and extraction routines. We believe that these issues are fairly generic for numerical simulation software, and the issues we report on provide a blueprint for similar undertakings. We also believe that our experiences provide guidance towards the development of AD interfaces that support AD tool interoperability. In addition, we improve upon the memory management of the existing ADOL-C instrumentation, which exhibited considerable runtime problems for higher mesh resolutions. We conduct serial and parallel ISSM model runs on a 2D mass transport benchmark as well as a model of the Pine Island Glacier to verify the derivatives computed by both tools and report on runtime performance and memory usage. In comparison, the CoDiPack AD variant of ISSM runs faster with less memory overhead than the ADOL-C variant and, thus, enables future model runs with an increased number of mesh elements. But the existence of two different AD implementations provides added confidence in the correctness of derivatives, in particular for future AD tool versions.

[1]  Roger P. Pawlowski,et al.  Automating embedded analysis capabilities and managing software complexity in multiphysics simulation, Part I: Template-based generic programming , 2012, Sci. Program..

[2]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[3]  Christian H. Bischof,et al.  On Combining Computational Differentiation and Toolkits for Parallel Scientific Computing , 2000, Euro-Par.

[4]  Eric Rignot,et al.  Optimal numerical solvers for transient simulations of ice flow using the Ice Sheet System Model (ISSM versions 4.2.5 and 4.11) , 2016, Geoscientific Model Development.

[5]  Jean Utke,et al.  Toward adjoinable MPI , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[6]  Sri Hari Krishna Narayanan,et al.  A Mixed Approach to Adjoint Computation with Algorithmic Differentiation , 2015, System Modelling and Optimization.

[7]  Uwe Naumann,et al.  Algorithmic Differentiation of a Complex C++ Code with Underlying Libraries , 2013, ICCS.

[8]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[9]  Jean Utke,et al.  An approach to computing discrete adjoints for MPI-parallelized models applied to Ice Sheet System Model 4.11 , 2016 .

[10]  Christian H. Bischof,et al.  Checking C++ codes for compatibility with operator overloading , 2015, 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[11]  Eric Rignot,et al.  Continental scale, high order, high spatial resolution, ice sheet modeling using the Ice Sheet System Model (ISSM) , 2012 .

[12]  Christian H. Bischof,et al.  Implementation of automatic differentiation tools , 2002, PEPM '02.

[13]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[14]  Nicholas Nethercote,et al.  Valgrind: a framework for heavyweight dynamic binary instrumentation , 2007, PLDI '07.

[15]  J. Utke,et al.  Inferred basal friction and surface mass balance of the Northeast Greenland Ice Stream using data assimilation of ICESat (Ice Cloud and land Elevation Satellite) surface altimetry and ISSM (Ice Sheet System Model) , 2014 .

[16]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[17]  M. Giles.  Collected Matrix Derivative Results for Forward and Reverse Mode Algorithmic Differentiation , 2008 .

[18]  Uwe Naumann,et al.  Algorithmic Differentiation of Numerical Methods , 2015, ACM Trans. Math. Softw..

[19]  A. Keane,et al.  Adjoint differentiation of a structural dynamics solver , 2006 .

[20]  Thomas Kaminski,et al.  Recipes for adjoint code construction , 1998, TOMS.

[21]  Johannes Willkomm,et al.  The Impact of Dynamic Data Reshaping on Adjoint Code Generation for Weakly-Typed Languages Such as Matlab , 2012 .

[22]  Patrick Amestoy,et al.  MUMPS : A General Purpose Distributed Memory Sparse Solver , 2000, PARA.

[23]  F. Potra,et al.  Sensitivity analysis for atmospheric chemistry models via automatic differentiation , 1997 .

[24]  U. Naumann,et al.  dco/c++: Derivative Code by Overloading in C++ , 2011 .

[25]  M. Morlighem,et al.  Sensitivity of the dynamics of Pine Island Glacier, West Antarctica, to climate forcing for the next 50 years , 2014 .

[26]  Robin J. Hogan,et al.  Fast Reverse-Mode Automatic Differentiation using Expression Templates in C++ , 2014, ACM Trans. Math. Softw..

[27]  Paul D. Hovland,et al.  Challenges and Opportunities in Using Automatic Differentiation with Object-Oriented Toolkits for Scientific Computing , 2003 .

[28]  Uwe Naumann,et al.  The Art of Differentiating Computer Programs - An Introduction to Algorithmic Differentiation , 2012, Software, environments, tools.

[29]  Christian H. Bischof,et al.  Sensitivity Analysis of Turbulence Models Using Automatic Differentiation , 2005, SIAM J. Sci. Comput..

[30]  Beckett Y. Zhou,et al.  A Discrete Adjoint Framework for Unsteady Aerodynamic and Aeroacoustic Optimization , 2015 .

[31]  Boyana Norris,et al.  ADIC2: Development of a component source transformation system for differentiating C and C++ , 2010, ICCS.

[32]  Uwe Naumann,et al.  "To be recorded" analysis in reverse-mode automatic differentiation , 2005, Future Gener. Comput. Syst..

[33]  Mathieu Morlighem,et al.  Ice sheet properties inferred by combining numerical modeling and remote sensing data , 2011 .

[34]  Mohammad R. Haghighat,et al.  Hierarchical approaches to automatic differentiation , 1996 .

[35]  Todd L. Veldhuizen,et al.  Expression templates , 1996 .

[36]  Jean Utke,et al.  Interpretative adjoints for numerical simulation codes using MPI , 2010, ICCS.

[37]  Christian H. Bischof,et al.  Source Transformation of C++ Codes for Compatibility with Operator Overloading , 2016, ICCS.

[38]  Roger P. Pawlowski,et al.  Automating embedded analysis capabilities and managing software complexity in multiphysics simulation, Part I: Template-based generic programming , 2012 .

[39]  Andrea Walther,et al.  Getting Started with ADOL-C , 2009, Combinatorial Scientific Computing.

[40]  M. E. Galassi,et al.  GNU SCIENTI C LIBRARY REFERENCE MANUAL , 2005 .

[41]  Cédric Jamet,et al.  Data Assimilation Methods , 2013 .

[42]  N. Gauger,et al.  Development of a Consistent Discrete Adjoint Solver in an Evolving Aerodynamic Design Framework , 2015 .