Answer
$$\frac{1}{6}\ln \left| {\frac{{x - 2}}{{x + 2}}} \right| + \frac{1}{{3\sqrt 2 }}{\tan ^{ - 1}}\left( {\frac{x}{{\sqrt 2 }}} \right) + C$$
Work Step by Step
$$\eqalign{
& \int {\frac{{{x^2}}}{{{x^4} - 2{x^2} - 8}}} dx \cr
& {\text{Factor the denominator}} \cr
& = \int {\frac{{{x^2}}}{{\left( {{x^2} - 4} \right)\left( {{x^2} + 2} \right)}}} dx \cr
& = \int {\frac{{{x^2}}}{{\left( {x + 2} \right)\left( {x - 2} \right)\left( {{x^2} + 2} \right)}}} dx \cr
& {\text{Decompose the integrand into partial fractions}} \cr
& \frac{{{x^2}}}{{\left( {x + 2} \right)\left( {x - 2} \right)\left( {{x^2} + 2} \right)}} = \frac{A}{{x + 2}} + \frac{B}{{x - 2}} + \frac{{Cx + D}}{{{x^2} + 2}} \cr
& {x^2} = A\left( {x - 2} \right)\left( {{x^2} + 2} \right) + B\left( {x + 2} \right)\left( {{x^2} + 2} \right) + \left( {Cx + D} \right)\left( {{x^2} - 4} \right) \cr
& {x^2} = A\left( {{x^3} + 2x - 2{x^2} - 4} \right) + B\left( {{x^3} + 2x + 2{x^2} + 4} \right) \cr
& + C{x^3} - 4Cx + D{x^2} - 4D \cr
& {x^2} = A{x^3} + 2Ax - 2A{x^2} - 4A + B{x^3} + 2Bx + 2B{x^2} + 4B \cr
& + C{x^3} - 4Cx + D{x^2} - 4D \cr
& {\text{Group terms}} \cr
& {x^2} = \left( {A + B + C} \right){x^3} + \left( { - 2A + 2B + D} \right){x^2} + \left( {2A + 2B - 4C} \right)x \cr
& - 4A + 4B - 4D \cr
& A + B + C = 0 \cr
& - 2A + 2B + D = 1 \cr
& 2A + 2B - 4C = 0 \cr
& - 4A + 4B - 4D = 0 \cr
& {\text{Solving the system of equations by using a calculator we have}} \cr
& A = - \frac{1}{6},{\text{ }}B = \frac{1}{6},{\text{ }}C = 0,{\text{ }}D = \frac{1}{3} \cr
& \frac{{{x^2}}}{{\left( {x + 2} \right)\left( {x - 2} \right)\left( {{x^2} + 2} \right)}} = \frac{{ - 1/6}}{{x + 2}} + \frac{{1/6}}{{x - 2}} + \frac{{0x + \frac{1}{3}}}{{{x^2} + 2}} \cr
& \int {\frac{{{x^2}}}{{{x^4} - 2{x^2} - 8}}} dx = \int {\left( {\frac{{ - 1/6}}{{x + 2}} + \frac{{1/6}}{{x - 2}} + \frac{{\frac{1}{3}}}{{{x^2} + 2}}} \right)} dx \cr
& {\text{Integrating}} \cr
& = - \frac{1}{6}\ln \left| {x + 2} \right| + \frac{1}{6}\ln \left| {x - 2} \right| + \frac{1}{{3\sqrt 2 }}{\tan ^{ - 1}}\left( {\frac{x}{{\sqrt 2 }}} \right) + C \cr
& = \frac{1}{6}\ln \left| {\frac{{x - 2}}{{x + 2}}} \right| + \frac{1}{{3\sqrt 2 }}{\tan ^{ - 1}}\left( {\frac{x}{{\sqrt 2 }}} \right) + C \cr} $$
