Answer
$$\int x^3\sqrt{x^2+1}dx=\frac{1}{5}(x^2+1)^{5/2}-\frac{1}{3}(x^2+1)^{3/2}+C$$
Work Step by Step
$$A=\int x^3\sqrt{x^2+1}dx=\int x^2\sqrt{x^2+1}xdx$$
We set $u=x^2+1$, which means $x^2=u-1$
Then $$du=2xdx$$ $$xdx=\frac{1}{2}du$$
Therefore, $$A=\frac{1}{2}\int(u-1)\sqrt udu=\frac{1}{2}\int\Big((u-1)u^{1/2}\Big)du$$ $$A=\frac{1}{2}\int\Big(u^{3/2}-u^{1/2}\Big)du$$ $$A=\frac{1}{2}\Big(\frac{u^{5/2}}{\frac{5}{2}}-\frac{u^{3/2}}{\frac{3}{2}}\Big)+C$$ $$A=\frac{u^{5/2}}{5}-\frac{u^{3/2}}{3}+C$$ $$A=\frac{1}{5}(x^2+1)^{5/2}-\frac{1}{3}(x^2+1)^{3/2}+C$$
