Refine
Language
- English (7)
Keywords
- TR1 Update (1)
- adjoint based updating (1)
- convergence order (1)
- local convergence (1)
- nonlinear equations (1)
- quai-Newton (1)
- quasi-Newton methods (1)
Project
- C12 (3)
Application Area
- C (3)
Quasi-Newton methods based on least change secant updating
formulas that solve linear equations $Ax=b$ in $n=\dim(x)=\dim(b)$ steps
can be expected to solve corresponding smooth nonlinear
systems $n$-step quadratically, i.e. with an $r$-order
of $\rho = 2^{1/n} = 1 + 1/n +O(1/n^2)$. The best rate one can
possibly expect on general problems is given by the positive root
$\rho_n$ of $\rho^n(\rho -1)=1$, for which
$\rho_n-1 = \ln(n)/n + O(1/n^2)$. To show that this upper bound is
actually achieved one usually has to impose a priori some kind of
linear independence condition on the sequence of steps taken by the
quasi-Newton iteration in question. Without any such assumptions we
establish in this paper the convergence order $\rho_n$ for the
two-sided rank one formula proposed by Schlenkrich et al in \cite{SGW06}.
It requires the evaluation of adjoint vectors, is invariant with respect
to linear transformations on the variable domain and combines the
properties of bounded deterioration and heredity.
For the solution of nonlinear equation systems
quasi-Newton methods based on low-rank updates are of particular interest. We analyze a class
of TR1 update formulas to approximate the system Jacobian. The local q-superlinear convergence for nonlinear problems is proved for a particular subclass of updates. Moreover, we give an estimate of the r-order of convergence. Numerical results comparing the TR1 method to Newton's and other quasi-Newton methods atr presented.
On the Efficient Generation of Taylor Expansions for DAE Solutions by Automatic Differentiation
(2005)
Under certain conditions the signature method suggested by
Pantiledes and Pryce facilitates the local expansion of DAE solutions
by Taylor polynomials of arbitrary order. The successive calculation of
Taylor coefficients involves the solution of nonlinear algebraic equations
by some variant of the Gauss-Newton method. Hence, one needs to evaluate
certain Jacobians and several right hand sides. Without advocating
a particular solver we discuss how this information can be efficiently
obtained using ADOL-C or similar automatic differentiation packages.3
Systems of stiff ordinary differential equations (ODEs) can be integrated
properly only by implicit methods. For that purpose, one usually has to
solve a system of nonlinear equations at each time step. This system of equations
may be solved by variants of Newton's method. Here, the main computing effort lies
in forming and factoring the Jacobian or a suitable approximation to it.
In this paper, we examine a new approach of constructing an appropriate quasi-
Newton approximation for solving stiff ODEs. The method makes for the first time
explicit use of tangent and adjoint information that can be obtained using the forward
and the reverse mode of algorithmic differentiation (AD). We elaborate the
conditions for invariance with respect to linear transformations of the state space
and thus similarity transformations of the Jacobian. One new updating variant that
yields such an invariant method is presented. Numerical results for Runge-Kutta
methods and linear multi-step methods are discussed.
Adjoint Broyden a la GMRES
(2007)
It is shown here that a compact storage implementation of a quasi-Newton
method based on the adjoint Broyden update reduces in the affine
case exactly to the well established GMRES procedure. Generally,
storage and linear algebra effort per step are small multiples of $n\cdot k$,
where $n$ is the number of variables and $k$ the number of steps taken
in the current cycle. In the affine case the storage is exactly $(n+k)\cdot k$
and in the nonlinear case the same bound can be achieved if adjoints,
i.e. transposed Jacobian-vector products are available. A
transposed-free variant that relies exclusively on Jacobian-vector
products (or possibly their approximation by divided differences)
requires roughly twice the storage and turns out to be somewhat slower
in our numerical experiments reported at the end.