Appendix B

To perform a line minimization from a point $X^{(k)}$ along a certain direction $D^{(k)}$, we wish to find $\lambda_{\mathrm{opt}}$, the optimum value of $\lambda$ which minimizes

$$f(\lambda) = \sum_{m=1}^{M} \frac{ (X^{(k)}_m + \lambda D^{(... ...}_m + \lambda D^{(k)}_m)^T S (X^{(k)}_m + \lambda D^{(k)}_m)}.$$ (27)

This may be achieved in several ways. First, by calculating the derivative of $f$ at $\lambda = 0$, $\left. \frac{\d f}{\d\lambda} \right\vert _{\lambda = 0}$, taking a trial step $\lambda_{\mathrm{t}}$ to evaluate $f_{\mathrm{t}} = f(\lambda_{\mathrm{t}})$ and making a parabolic fit to determine $\lambda_{\mathrm{opt}}$.

Alternatively, since

$$\frac{\d f}{\d\lambda} = \sum_{m=1}^{M} \frac{2(a_m + \lambda b_m + \lambda^2 c_m)}{(1+ \lambda^2 (D_m)^T S D_m)^2},$$ (28)

where

$$a_m = \left[\left(X^{(k)}_m\right)^T S X^{(k)}_m\right] \left[\left(D^{(k)}_m\right)^T H X^{(k)}_m\right] -$$

$$\qquad\qquad \left[\left(X^{(k)}_m\right)^T H X^{(k)}_m\right] \left[\left(D^{(k)}_m\right)^T SX^{(k)}_m\right],$$ (29)

$$b_m = \left[\left(X^{(k)}_m \right)^T SX^{(k)}_m\right] \left[\left(D^{(k)}_m \right)^T HD^{(k)}_m\right] -$$

$$\qquad\qquad \left[\left(X^{(k)}_m \right)^T H X^{(k)}_m \right] \left[\left(D^{(k)}_m \right)^T SD^{(k)}_m \right],$$ (30)

$$c_m = \left[\left(X^{(k)}_m \right)^T SD^{(k)}_m \right] \left[\left(D^{(k)}_m \right)^T H D^{(k)}_m \right] -$$

$$\qquad\qquad \left[\left(X^{(k)}_m \right)^T H D^{(k)}_m \right] \left[\left(D^{(k)}_m \right)^T S D^{(k)}_m \right],$$ (31)

we find $\lambda_{\mathrm{opt}}$ as one of the roots of the quadratic equation

$$a + b \lambda + c \lambda^2 = 0,$$ (32)

where $a = {\displaystyle{\sum_m} a_m}$ etc.