Optimization · Definition

Hessian matrix

multivariable calculusoptimization

Before you start you need to know

It is the multivariable analogue of the second derivative: it stores the curvature of \(f\) in every direction and lets us classify critical points.

Definition

The Hessian matrix \(H\) of a function \(f\) of two variables is the \(2\times2\) matrix of its second-order partial derivatives at a point.

\[H=\begin{pmatrix}f_{xx}&f_{xy}\\ f_{yx}&f_{yy}\end{pmatrix}\]

To compute it, differentiate the partials a second time. If \(f(x,y)=x^3y\), then \(f_x=3x^2y\), \(f_y=x^3\) and \(f_{xx}=6xy\), \(f_{xy}=3x^2\), \(f_{yy}=0\); at \((1,1)\) the Hessian is \(\begin{pmatrix}6&3\\3&0\end{pmatrix}\). The notation \(f_{xy}\) means differentiating first with respect to \(x\) and then \(y\), and books differ on the order.

If the second derivatives are continuous, Clairaut's theorem gives \(f_{xy}=f_{yx}\), so \(H\) is symmetric and only three distinct entries need computing. This works as a check: if your two mixed partials disagree, there is a computational error.

Its determinant \(D=f_{xx}f_{yy}-f_{xy}^{2}\) and the entry \(f_{xx}\) are the data used by the second derivative test. A negative determinant means curvatures of opposite sign in two directions, that is, a saddle point.

For three or more variables the idea is the same: the Hessian is the \(n\times n\) matrix of all partials \(f_{ij}\). In one variable it reduces to \(f''\), which is why it is seen as the second derivative in matrix form.

Think about it

For \(f(x,y)=x^2y^2\), why is the Hessian at \((0,0)\) the zero matrix even though \(f_{xy}=4xy\) is not identically zero?

Because the Hessian is evaluated at a point: at the origin \(f_{xx}=2y^2=0\), \(f_{xy}=4xy=0\) and \(f_{yy}=2x^2=0\). Away from the origin the entries change with the point.

Exercises 10

  1. Compute the Hessian matrix of \(f(x,y)=x^2+3xy+y^2\).
  2. Compute the Hessian of \(f(x,y)=x^3y\) at \((1,1)\).
  3. Compute \(\det H\) for \(f(x,y)=x^2+xy+y^2\).
  4. Compute the Hessian of \(f(x,y)=e^{xy}\) at \((0,0)\).
  5. Compute the Hessian of \(f(x,y)=x\ln y\) at \((1,1)\).
  6. If \(f\) has continuous second-order partial derivatives, what always holds for its Hessian \(H\)?
  7. Compute the trace of the Hessian of \(f(x,y)=x^3+y^3-3xy\) at \((1,1)\).
  8. For \(k>0\), which value makes the Hessian of \(f(x,y)=x^2+kxy+y^2\) singular?
  9. Compute \(\det H\) at \((1,1)\) for \(f(x,y)=x^2y^2\).
  10. What size is the Hessian of a function \(f(x,y,z)\) of three variables?

References

  • James Stewart, Daniel K. Clegg, Saleem Watson, Calculus: Early Transcendentals, 9th edition, Cengage, 2021. §14.7
  • Joel Hass, Christopher Heil, Przemyslaw Bogacki, Maurice D. Weir, George B. Thomas Jr., Thomas' Calculus, 15th edition, Pearson, 2023. §14.7
  • Jerrold E. Marsden, Anthony Tromba, Vector Calculus, 6th edition, W. H. Freeman, 2012. §3.3