[tex]f(x,y)=xy-x^2-y^2-3x-3y+12[/tex]
First compute the first-order partial derivatives and find the critical points.
[tex]f_x=y-2x-3[/tex]
[tex]f_y=x-2y-3[/tex]
Both first order derivatives vanish at [tex](x,y)=(-3,-3)[/tex].
Computing the Hessian, we get
[tex]\mathbf H(x,y)=\begin{bmatrix}f_{xx}&f_{xy}\\f_{yx}&f_{yy}\end{bmatrix}=\begin{bmatrix}-2&1\\1&-2\end{bmatrix}[/tex]
We have [tex]\det\mathbf H(x,y)=3>0[/tex], which means [tex](-3,-3)[/tex] is an extremum of [tex]f(x,y)[/tex]. Since [tex]f_{xx}(-3,-3)=-2<0[/tex], this extremum is a local maximum of [tex]f(x,y)[/tex] with a value of 21.
