Answer
$\dfrac{1+x^4y^2+y^2+x^4y^4-2xy}{x^2-2xy-2x^5y^3}$
Work Step by Step
We are given that $ \tan^{-1} (x^2y)=x+xy^2$
Re-arrange as: $\tan^{-1} (x^2y)-x-xy^2=0$
Consider $F(x,y)=\tan^{-1} (x^2y)-x-xy^2=0$
$F_x=\dfrac{(2xy)}{1+(x^2y)^2} -1-y^2$ and $F_y=\dfrac{x^2}{1+(x^2y)^2} -2xy$
Use Equation 6, which is: $\dfrac{dy}{dx}=-\dfrac{F_x}{F_y}$
This implies that
$\dfrac{dy}{dx}=-\dfrac{\dfrac{(2xy)}{1+(x^2y)^2} -1-y^2}{\dfrac{x^2}{1+(x^2y)^2} -2xy}=\dfrac{(1+y^2)(1+x^4y^2)-2xy}{x^2-2xy(1+x^4y^2)}=\dfrac{1+x^4y^2+y^2+x^4y^4-2xy}{x^2-2xy-2x^5y^3}$
