[tex]f(x,y)=9+xy-x-2y[/tex]
[tex]\implies\begin{cases}f_x=y-1=0\implies y=1\\f_y=x-2=0\implies x=2\end{cases}[/tex]
which tells you that the only critical point of [tex]f(x,y)[/tex] occurs at (2, 1), which does lie within the region [tex]D[/tex]. At this point, we get [tex]f(2,1)=7[/tex].
Next we check along the boundaries of [tex]D[/tex]. They are the lines [tex]x=1[/tex] with [tex]0\le y\le4[/tex], [tex]y=0[/tex] with [tex]1\le x\le5[/tex], and [tex]y=5-x[/tex] with [tex]1\le x\le5[/tex].
If [tex]x=1[/tex], then [tex]f(1,y)=8-y[/tex], which is monotonically decreasing and must therefore attain its maximum at [tex]y=0[/tex] and minimum at [tex]y=4[/tex]. We get [tex]f(1,0)=8[/tex] and [tex]f(1,4)=4[/tex].
If [tex]y=0[/tex], then [tex]f(x,0)=9-x[/tex], which is also monotonically decreasing and attains its maximum at [tex]x=1[/tex] and minimum at [tex]x=5[/tex]. We get [tex]f(1,0)=8[/tex] and [tex]f(5,0)=4[/tex].
If [tex]y=5-x[/tex], then [tex]f(x,5-x)=g(x)=-x^2+6x-1[/tex]. We have [tex]g'(x)=-2x+6=0\implies x=3[/tex], which suggests an extremum occurs at (3, 2). We get [tex]f(3,2)=8[/tex].
So [tex]f(x,y)[/tex] has a minimum value of 4 at (1, 4) and (5, 0), and a maximum value of 8 at (1, 0) and (3, 2).
