The following technical report is available from http://aib.informatik.rwth-aachen.de:
Algorithmic Differentiation of Numerical Methods: Second-Order Tangent and Adjoint Solvers for Systems of Parametrized Nonlinear Equations Niloofar Safiran, Johannes Lotz, and Uwe Naumann AIB 2014-07
Forward and reverse modes of algorithmic differentiation (AD) transform implementations of multivariate vector functions as computer programs into tangent and adjoint code, respectively. The reapplication of the same ideas yields higher derivative code. In particular, second derivatives play an important role in nonlinear programming. Second-order methods based on Newton's algorithm promise faster convergence in the neighbourhood of the minimum by taking into account second derivative information. The adjoint mode is of particular interest in large-scale nonlinear optimization due to the independence of its computational cost on the number of free variables. Solvers for parametrized system of n equations embedded into evaluation of objective function for a (without loss of generality) unconstrained nonlinear optimization problem. Require Hessian of objective with respect to free variables implying need for second derivatives of the nonlinear solver. The local computational overhead as well as the additional memory requirement for the computation of second-order tangents/adjoints of the solution vector with respect to parameters by a fully discrete method (derived by AD) can quickly become prohibitive for large values of n. Both can be reduced extremely by the second-order continuous approach to differentiation of the underlying numerical method to be discussed in this paper.