Answer
$$\eqalign{
& \left( {\text{a}} \right)\frac{{2 - \sqrt 2 }}{3} \cr
& \left( {\text{b}} \right)\frac{{2 - \sqrt 2 }}{3} \cr} $$
Work Step by Step
$$\eqalign{
& \int_0^1 {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx \cr
& \left( {\text{a}} \right){\text{ Using integration by parts}} \cr
& \int {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx = \int {{x^2}\left( {\frac{x}{{\sqrt {{x^2} + 1} }}} \right)} dx \cr
& u = {x^2},{\text{ }}du = 2xdx \cr
& dv = \frac{x}{{\sqrt {{x^2} + 1} }},{\text{ }}v = \sqrt {{x^2} + 1} \cr
& \int {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx = {x^2}\sqrt {{x^2} + 1} - \int {\sqrt {{x^2} + 1} } \left( {2x} \right)dx \cr
& {\text{ }} = {x^2}\sqrt {{x^2} + 1} - \frac{{{{\left( {{x^2} + 1} \right)}^{3/2}}}}{{3/2}} + C \cr
& {\text{ }} = {x^2}\sqrt {{x^2} + 1} - \frac{{2{{\left( {{x^2} + 1} \right)}^{3/2}}}}{3} + C \cr
& {\text{Then}} \cr
& \int_0^1 {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx = \left[ {{x^2}\sqrt {{x^2} + 1} - \frac{{2{{\left( {{x^2} + 1} \right)}^{3/2}}}}{3}} \right]_0^1 \cr
& = \left[ {{1^2}\sqrt {{1^2} + 1} - \frac{{2{{\left( {{1^2} + 1} \right)}^{3/2}}}}{3}} \right] - \left[ {{0^2}\sqrt {{0^2} + 1} - \frac{{2{{\left( {{0^2} + 1} \right)}^{3/2}}}}{3}} \right] \cr
& = \sqrt 2 - \frac{2}{3}{\left( 2 \right)^{3/2}} + \frac{2}{3} \cr
& = \sqrt 2 - \frac{4}{3}\sqrt 2 + \frac{2}{3} \cr
& = - \frac{1}{3}\sqrt 2 + \frac{2}{3} \cr
& = \frac{{2 - \sqrt 2 }}{3} \cr
& \cr
& \left( {\text{b}} \right){\text{ Using the substitution }}u = \sqrt {{x^2} + 1} \cr
& {u^2} = {x^2} + 1 \cr
& 2udu = 2xdx \cr
& udu = xdx \cr
& \int {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx = \int {\frac{{{u^2} - 1}}{u}} \left( {udu} \right) \cr
& {\text{ }} = \int {\left( {{u^2} - 1} \right)} du = \frac{1}{3}{u^3} - u + C \cr
& {\text{ }} = \frac{1}{3}{\left( {\sqrt {{x^2} + 1} } \right)^3} - \sqrt {{x^2} + 1} + C \cr
& \int_0^1 {\frac{{{x^3}}}{{\sqrt {{x^2} + 1} }}} dx = \left[ {\frac{1}{3}{{\left( {\sqrt {{x^2} + 1} } \right)}^3} - \sqrt {{x^2} + 1} } \right]_0^1 \cr
& = \left[ {\frac{1}{3}{{\left( {\sqrt {{1^2} + 1} } \right)}^3} - \sqrt {{1^2} + 1} } \right] - \left[ {\frac{1}{3}{{\left( {\sqrt {{0^2} + 1} } \right)}^3} - \sqrt {{0^2} + 1} } \right] \cr
& = \frac{1}{3}{\left( {\sqrt 2 } \right)^3} - \sqrt 2 - 1 \cr
& = \frac{2}{3}\sqrt 2 - \sqrt 2 - 1 \cr
& = \frac{{2 - \sqrt 2 }}{3} \cr} $$