Answer
$$\int x^3\sqrt{x^2+1}dx=\frac{(\sqrt{x^2+1})^5}{5}-\frac{(\sqrt{x^2+1})^3}{3}+C$$
Work Step by Step
$$A=\int x^3\sqrt{x^2+1}dx$$
Let $a=\sqrt{x^2+1}$. That means $x^2+1=a^2$ and $x^2=a^2-1$
Also, $da=\frac{(x^2+1)'}{2\sqrt{x^2+1}}dx=\frac{2x}{2\sqrt{x^2+1}}dx=\frac{x}{\sqrt{x^2+1}}dx=\frac{x}{a}dx$
which means $xdx=ada$
Therefore, if we rewrite $A$ into $$A=\int x^2\sqrt{x^2+1}xdx$$
we would have $$A=\int(a^2-1)\times a\times ada$$ $$A=\int(a^4-a^2)da$$ $$A=\frac{a^5}{5}-\frac{a^3}{3}+C$$ $$A=\frac{(\sqrt{x^2+1})^5}{5}-\frac{(\sqrt{x^2+1})^3}{3}+C$$
