report/numJac.tex

   1 \documentclass[a4paper,10pt]{article}
   2
   3 \title{Numerical Jacobian evaluation}
   4 \author{Antonio, Fabio Di Narzo}
   5
   6 \begin{document}
   7 \maketitle
   8
   9 Consider the map $F: \Re^p \mapsto \Re^p$:
  10 \[
  11 F(\mathbf{x}) = \left( \begin{array}{c}
  12 f_1(\mathbf{x}) \\
  13 f_2(\mathbf{x}) \\
  14 \ldots\\
  15 f_p(\mathbf{x})
  16 \end{array} \right)
  17 \]
  18 with $\mathbf{x} \in \Re^p$. Define $D_j F$ as the column vector:
  19 \[
  20 D_j F = \left( \begin{array}{c}
  21 D_j f_1 \\
  22 D_j f_2 \\
  23 \ldots\\
  24 D_j f_p
  25 \end{array}
  26 \right)
  27 \]
  28 with $D_j f_i$ indicating the derivative of $f_i$ with respect to the $j$-th component of $\mathbf{x}$.
  29 We want to evaluate the Jacobian:
  30 \[
  31 JF = \left[ \begin{array}{cccc} D_1F & D_2F & \ldots & D_pF \end{array} \right] =
  32 \left( \begin{array}{cccc}
  33 D_1 f_1 & D_2 f_1 & \ldots & D_p f_1 \\
  34 D_1 f_2 & D_2 f_2 & \ldots & D_p f_2 \\
  35 \ldots \\
  36 D_1 f_p & D_2 f_p & \ldots & D_p f_p \\
  37 \end{array}\right)
  38 \]
  39 The computation of the matrix $JF$ reduces to the computation of the $p \times p$ partial derivatives $D_j f_i$ in a
  40 generic point $\mathbf{x}$. A numerical approximation of such derivative can be the following:
  41 \[
  42 D_j f_i (\mathbf{x}) \simeq \frac{1}{h} [f_i(\mathbf{x}^{(j)\star}) - f_i(\mathbf{x})]
  43 \]
  44 with $ \mathbf{x}^{(j)\star} = (x_1, x_2, \ldots, x_j+h, \ldots, x_p)$ and $h \simeq 0$ \footnote{
  45 The increment $h$ should be chosen as the smallest positive value for which $x_j+h \neq x_j$ on the computer.
  46 It can be worth to define $h$ relative to each $x_j$, i.e. $h = x_j \times h^\star$
  47  with $h^\star \simeq 0$ fixed once for all $j$.
  48 }.
  49
  50 Note that the actual computation of these $p \times p$ derivatives requires $p+1$ evaluations of the map $F$.
  51 This can be seen by using matrix notation:
  52 \[
  53 D_jF(\mathbf{x}) \simeq  \frac{1}{h} [F(\mathbf{x}^{(j)\star}) - F(\mathbf{x})]
  54 \]
  55 So, one can compute $F(\mathbf{x})$ once (stays fixed over $j$) and then the $p$ vectors $F(\mathbf{x}^{(j)\star}),
  56 \quad 1\leq j\leq p$.
  57
  58 \end{document}