Fréchet derivative

(Redirected from Frechet derivative)

In mathematics, the Fréchet derivative is a derivative defined on normed spaces. Named after Maurice Fréchet, it is commonly used to generalize the derivative of a real-valued function of a single real variable to the case of a vector-valued function of multiple real variables, and to define the functional derivative used widely in the calculus of variations.

Generally, it extends the idea of the derivative from real-valued functions of one real variable to functions on normed spaces. The Fréchet derivative should be contrasted to the more general Gateaux derivative which is a generalization of the classical directional derivative.

The Fréchet derivative has applications to nonlinear problems throughout mathematical analysis and physical sciences, particularly to the calculus of variations and much of nonlinear analysis and nonlinear functional analysis.

Definition

edit

Let   and   be normed vector spaces, and   be an open subset of   A function   is called Fréchet differentiable at   if there exists a bounded linear operator   such that  

The limit here is meant in the usual sense of a limit of a function defined on a metric space (see Functions on metric spaces), using   and   as the two metric spaces, and the above expression as the function of argument   in   As a consequence, it must exist for all sequences   of non-zero elements of   that converge to the zero vector   Equivalently, the first-order expansion holds, in Landau notation  

If there exists such an operator   it is unique, so we write   and call it the Fréchet derivative of   at   A function   that is Fréchet differentiable for any point of   is said to be C1 if the function   is continuous (  denotes the space of all bounded linear operators from   to  ). Note that this is not the same as requiring that the map   be continuous for each value of   (which is assumed; bounded and continuous are equivalent).

This notion of derivative is a generalization of the ordinary derivative of a function on the real numbers   since the linear maps from   to   are just multiplication by a real number. In this case,   is the function  

Properties

edit

A function differentiable at a point is continuous at that point.

Differentiation is a linear operation in the following sense: if   and   are two maps   which are differentiable at   and   is a scalar (a real or complex number), then the Fréchet derivative obeys the following properties:    

The chain rule is also valid in this context: if   is differentiable at   and   is differentiable at   then the composition   is differentiable in   and the derivative is the composition of the derivatives:  

Finite dimensions

edit

The Fréchet derivative in finite-dimensional spaces is the usual derivative. In particular, it is represented in coordinates by the Jacobian matrix.

Suppose that   is a map,   with   an open set. If   is Fréchet differentiable at a point   then its derivative is   where  denotes the Jacobian matrix of   at  

Furthermore, the partial derivatives of   are given by   where   is the canonical basis of   Since the derivative is a linear function, we have for all vectors   that the directional derivative of   along   is given by  

If all partial derivatives of   exist and are continuous, then   is Fréchet differentiable (and, in fact, C1). The converse is not true; the function   is Fréchet differentiable and yet fails to have continuous partial derivatives at  

Example in infinite dimensions

edit

One of the simplest (nontrivial) examples in infinite dimensions, is the one where the domain is a Hilbert space ( ) and the function of interest is the norm. So consider  

First assume that   Then we claim that the Fréchet derivative of   at   is the linear functional   defined by  

Indeed,  

Using continuity of the norm and inner product we obtain:  

As   and because of the Cauchy-Schwarz inequality   is bounded by   thus the whole limit vanishes.

Now we show that at   the norm is not differentiable, that is, there does not exist bounded linear functional   such that the limit in question to be   Let   be any linear functional. Riesz Representation Theorem tells us that   could be defined by   for some   Consider  

In order for the norm to be differentiable at   we must have  

We will show that this is not true for any   If   obviously   independently of   hence this is not the derivative. Assume   If we take   tending to zero in the direction of   (that is,   where  ) then   hence  

(If we take   tending to zero in the direction of   we would even see this limit does not exist since in this case we will obtain  ).

The result just obtained agrees with the results in finite dimensions.

Relation to the Gateaux derivative

edit

A function   is called Gateaux differentiable at   if   has a directional derivative along all directions at   This means that there exists a function   such that   for any chosen vector   and where   is from the scalar field associated with   (usually,   is real).[1]

If   is Fréchet differentiable at   it is also Gateaux differentiable there, and   is just the linear operator  

However, not every Gateaux differentiable function is Fréchet differentiable. This is analogous to the fact that the existence of all directional derivatives at a point does not guarantee total differentiability (or even continuity) at that point. For example, the real-valued function   of two real variables defined by   is continuous and Gateaux differentiable at the origin  , with its derivative at the origin being  

The function   is not a linear operator, so this function is not Fréchet differentiable.

More generally, any function of the form   where   and   are the polar coordinates of   is continuous and Gateaux differentiable at   if   is differentiable at   and   but the Gateaux derivative is only linear and the Fréchet derivative only exists if   is sinusoidal.

In another situation, the function   given by   is Gateaux differentiable at   with its derivative there being   for all   which is a linear operator. However,   is not continuous at   (one can see by approaching the origin along the curve  ) and therefore   cannot be Fréchet differentiable at the origin.

A more subtle example is   which is a continuous function that is Gateaux differentiable at   with its derivative at this point being   there, which is again linear. However,   is not Fréchet differentiable. If it were, its Fréchet derivative would coincide with its Gateaux derivative, and hence would be the zero operator  ; hence the limit   would have to be zero, whereas approaching the origin along the curve   shows that this limit does not exist.

These cases can occur because the definition of the Gateaux derivative only requires that the difference quotients converge along each direction individually, without making requirements about the rates of convergence for different directions. Thus, for a given ε, although for each direction the difference quotient is within ε of its limit in some neighborhood of the given point, these neighborhoods may be different for different directions, and there may be a sequence of directions for which these neighborhoods become arbitrarily small. If a sequence of points is chosen along these directions, the quotient in the definition of the Fréchet derivative, which considers all directions at once, may not converge. Thus, in order for a linear Gateaux derivative to imply the existence of the Fréchet derivative, the difference quotients have to converge uniformly for all directions.

The following example only works in infinite dimensions. Let   be a Banach space, and   a linear functional on   that is discontinuous at   (a discontinuous linear functional). Let  

Then   is Gateaux differentiable at   with derivative   However,   is not Fréchet differentiable since the limit   does not exist.

Higher derivatives

edit

If   is a differentiable function at all points in an open subset   of   it follows that its derivative   is a function from   to the space   of all bounded linear operators from   to   This function may also have a derivative, the second order derivative of   which, by the definition of derivative, will be a map  

To make it easier to work with second-order derivatives, the space on the right-hand side is identified with the Banach space   of all continuous bilinear maps from   to   An element   in   is thus identified with   in   such that for all    

(Intuitively: a function   linear in   with   linear in   is the same as a bilinear function   in   and  ).

One may differentiate   again, to obtain the third order derivative, which at each point will be a trilinear map, and so on. The  -th derivative will be a function   taking values in the Banach space of continuous multilinear maps in   arguments from   to   Recursively, a function   is   times differentiable on   if it is   times differentiable on   and for each   there exists a continuous multilinear map   of   arguments such that the limit   exists uniformly for   in bounded sets in   In that case,   is the  st derivative of   at  

Moreover, we may obviously identify a member of the space   with a linear map   through the identification   thus viewing the derivative as a linear map.

Partial Fréchet derivatives

edit

In this section, we extend the usual notion of partial derivatives which is defined for functions of the form   to functions whose domains and target spaces are arbitrary (real or complex) Banach spaces. To do this, let   and   be Banach spaces (over the same field of scalars), and let   be a given function, and fix a point   We say that   has an i-th partial differential at the point   if the function   defined by

  is Fréchet differentiable at the point   (in the sense described above). In this case, we define   and we call   the i-th partial derivative of   at the point   It is important to note that   is a linear transformation from   into   Heuristically, if   has an i-th partial differential at   then   linearly approximates the change in the function   when we fix all of its entries to be   for   and we only vary the i-th entry. We can express this in the Landau notation as  

Generalization to topological vector spaces

edit

The notion of the Fréchet derivative can be generalized to arbitrary topological vector spaces (TVS)   and   Letting   be an open subset of   that contains the origin and given a function   such that   we first define what it means for this function to have 0 as its derivative. We say that this function   is tangent to 0 if for every open neighborhood of 0,   there exists an open neighborhood of 0,   and a function   such that   and for all   in some neighborhood of the origin,  

We can now remove the constraint that   by defining   to be Fréchet differentiable at a point   if there exists a continuous linear operator   such that   considered as a function of   is tangent to 0. (Lang p. 6)

If the Fréchet derivative exists then it is unique. Furthermore, the Gateaux derivative must also exist and be equal the Fréchet derivative in that for all     where   is the Fréchet derivative. A function that is Fréchet differentiable at a point is necessarily continuous there and sums and scalar multiples of Fréchet differentiable functions are differentiable so that the space of functions that are Fréchet differentiable at a point form a subspace of the functions that are continuous at that point. The chain rule also holds as does the Leibniz rule whenever   is an algebra and a TVS in which multiplication is continuous.

See also

edit

Notes

edit
  1. ^ It is common to include in the definition that the resulting map   must be a continuous linear operator. We avoid adopting this convention here to allow examination of the widest possible class of pathologies.

References

edit
  • Cartan, Henri (1967), Calcul différentiel, Paris: Hermann, MR 0223194.
  • Dieudonné, Jean (1969), Foundations of modern analysis, Boston, MA: Academic Press, MR 0349288.
  • Lang, Serge (1995), Differential and Riemannian Manifolds, Springer, ISBN 0-387-94338-2.
  • Munkres, James R. (1991), Analysis on manifolds, Addison-Wesley, ISBN 978-0-201-51035-5, MR 1079066.
  • Previato, Emma, ed. (2003), Dictionary of applied math for engineers and scientists, Comprehensive Dictionary of Mathematics, London: CRC Press, ISBN 978-1-58488-053-0, MR 1966695.
  • Coleman, Rodney, ed. (2012), Calculus on Normed Vector Spaces, Universitext, Springer, ISBN 978-1-4614-3894-6.
edit
  • B. A. Frigyik, S. Srivastava and M. R. Gupta, Introduction to Functional Derivatives, UWEE Tech Report 2008-0001.
  • http://www.probability.net. This webpage is mostly about basic probability and measure theory, but there is nice chapter about Frechet derivative in Banach spaces (chapter about Jacobian formula). All the results are given with proof.