Answer
(a) $f(x,y)$ is homogeneous function of degree $3$.
(b) \[x\frac{\partial f}{\partial x}+y\frac{\partial f}{\partial y}=nf\]
Work Step by Step
(a) Given that:
\[f(x,y)=x^2y+2xy^2+5y^3\]
\[f(tx,ty)=(tx)^2(ty)+2(tx)(ty)^2+5(ty)^3\]
\[f(tx,ty)=t^3x^2y+2t^3xy^2+5t^3y^3\]
\[f(tx,ty)=t^3(x^2y+2xy^2+5y^3)\]
\[\Rightarrow f(tx,ty)=t^3f(x,y)\]
Therefore $f(x,y)$ is homogeneous function of degree $3$.
(b) It is given that $f$ is homogeneous function of degree $n$
\[\Rightarrow f(x,y)=x^nf\left(1,\frac{y}{x}\right)\]
\[\Rightarrow f(x,y)=x^ng\left(\frac{y}{x}\right)\]
Differentiate $f$ with respect to $x$ treating $y$ as constant using chain rule:
\[\frac{\partial f}{\partial x}=nx^{n-1}g+x^{n}g'\left(\frac{y}{x}\right)\cdot\frac{\partial }{\partial x}\left(\frac{y}{x}\right)\]
\[\frac{\partial f}{\partial x}=nx^{n-1}g+x^{n}g'\left(\frac{y}{x}\right)\cdot\left(\frac{-y}{x^2}\right)\]
\[\frac{\partial f}{\partial x}=nx^{n-1}g-x^{n-2}yg'\left(\frac{y}{x}\right)\;\;\;\;\;\;\;\ldots (1)\]
Differentiate $f$ with respect to $y$ treating $x$ as constant using chain rule:
\[\frac{\partial f}{\partial y}=x^{n}g'\left(\frac{y}{x}\right)\cdot\frac{\partial }{\partial y}\left(\frac{y}{x}\right)\]
\[\frac{\partial f}{\partial y}=x^{n}g'\left(\frac{y}{x}\right)\cdot\left(\frac{1}{x}\right)\]
\[\frac{\partial f}{\partial y}=x^{n-1}g'\left(\frac{y}{x}\right)\;\;\;\;\;\;\;\;\ldots (2)\]
Consider $x\displaystyle\frac{\partial f}{\partial x}+y\displaystyle\frac{\partial f}{\partial y}$
Using (1) and (2)
\begin{eqnarray*}
x\displaystyle\frac{\partial f}{\partial x}+y\displaystyle\frac{\partial f}{\partial y}&=&x\left[nx^{n-1}g-x^{n-2}yg'\left(\frac{y}{x}\right)\right]+y\left[x^{n-1}g'\left(\frac{y}{x}\right)\right]\\
&=&nx^{n}g-x^{n-1}yg'\left(\frac{y}{x}\right)+x^{n-1}yg'\left(\frac{y}{x}\right)\\
&=&nx^ng\\
&=&nf(x,y)
\end{eqnarray*}
Hence Prove.
