Answer
(a) Solving the equations ${f_x} = 0$ and ${f_y} = 0$, we show that the critical points satisfy the following equations:
$y\left( {y - 2x + 1} \right) = 0$, ${\ \ }$ $x\left( {2y - x + 1} \right) = 0$
(b) $f$ has three critical points where $x=0$ or $y=0$ (or both) and one critical point where $x$ and $y$ are nonzero:
$\left( {0,0} \right)$,
$\left( {0, - 1} \right)$,
$\left( {1,0} \right)$, and
$\left( {\frac{1}{3}, - \frac{1}{3}} \right)$.
(c)
$\begin{array}{*{20}{c}}
{{\bf{Critical}}}&{{f_{xx}}}&{{f_{yy}}}&{{f_{xy}}}&{{\bf{Discriminant}}}&{}\\
{{\bf{Point}}}&{ - 2y}&{2x}&{2y - 2x + 1}&{D = {f_{xx}}{f_{yy}} - {f_{xy}}^2}&{{\bf{Type}}}\\
{\left( {0,0} \right)}&0&0&1&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {0, - 1} \right)}&2&0&{ - 1}&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {1,0} \right)}&0&2&{ - 1}&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {\frac{1}{3}, - \frac{1}{3}} \right)}&{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{1}{3}}&{{\rm{local{\ }minimum}}}
\end{array}$
Work Step by Step
(a) We have $f\left( {x,y} \right) = {y^2}x - y{x^2} + xy$.
The partial derivatives are
${f_x} = {y^2} - 2xy + y$, ${\ \ }$ ${f_y} = 2yx - {x^2} + x$
To find the critical points we solve the equations ${f_x} = 0$ and ${f_y} = 0$:
${y^2} - 2xy + y = 0$, ${\ \ }$ $2yx - {x^2} + x = 0$
$y\left( {y - 2x + 1} \right) = 0$, ${\ \ }$ $x\left( {2y - x + 1} \right) = 0$
Hence, the critical points $\left( {x,y} \right)$ satisfy the equations
(1) ${\ \ \ \ }$ $y\left( {y - 2x + 1} \right) = 0$, ${\ \ }$ $x\left( {2y - x + 1} \right) = 0$
(b) From the first equation of equation (1), we obtain
$y=0$, ${\ \ \ \ }$ $y=2x-1$
From the second equation of equation (1), we obtain
$x=0$, ${\ \ \ \ }$ $x=2y+1$
Substituting $y=0$ in $x=2y+1$; and $x=0$ in $y=2x-1$, we obtain the critical points: $\left( {0,0} \right)$, $\left( {0, - 1} \right)$, $\left( {1,0} \right)$.
Solving $y-2x+1=0$ and $2y-x+1=0$ simultaneously gives $x = \frac{1}{3}$ and $y = - \frac{1}{3}$.
So, we have another critical point $\left( {\frac{1}{3}, - \frac{1}{3}} \right)$.
Thus, the critical points of $f$ are $\left( {0,0} \right)$, $\left( {0, - 1} \right)$, $\left( {1,0} \right)$ and $\left( {\frac{1}{3}, - \frac{1}{3}} \right)$.
Hence, $f$ has three critical points where $x=0$ or $y=0$ (or both) and one critical point where $x$ and $y$ are nonzero.
(c) The second partial derivatives are
${f_{xx}} = - 2y$, ${\ \ }$ ${f_{yy}} = 2x$, ${\ \ }$ ${f_{xy}} = 2y - 2x + 1$
Using the Second Derivative Test, we determine the nature of the critical points and list the results in the following table:
$\begin{array}{*{20}{c}}
{{\bf{Critical}}}&{{f_{xx}}}&{{f_{yy}}}&{{f_{xy}}}&{{\bf{Discriminant}}}&{}\\
{{\bf{Point}}}&{ - 2y}&{2x}&{2y - 2x + 1}&{D = {f_{xx}}{f_{yy}} - {f_{xy}}^2}&{{\bf{Type}}}\\
{\left( {0,0} \right)}&0&0&1&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {0, - 1} \right)}&2&0&{ - 1}&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {1,0} \right)}&0&2&{ - 1}&{ - 1}&{{\rm{saddle{\ }point}}}\\
{\left( {\frac{1}{3}, - \frac{1}{3}} \right)}&{\frac{2}{3}}&{\frac{2}{3}}&{ - \frac{1}{3}}&{\frac{1}{3}}&{{\rm{local{\ }minimum}}}
\end{array}$
