Answer
$$3\sqrt {27} + \frac{9}{2}\ln \left( {6 + \sqrt {27} } \right) - 2\sqrt 7 - \frac{9}{2}\ln \left( {4 + \sqrt 7 } \right)$$
Work Step by Step
$$\eqalign{
& \int_4^6 {\frac{{{x^2}}}{{\sqrt {{x^2} - 9} }}} dx \cr
& {\text{Refer to the triangle below}} \cr
& \sec \theta = \frac{x}{3},{\text{ }}dx = 3\sec \theta \tan \theta d\theta \cr
& {\text{Substitute}} \cr
& \int {\frac{{{x^2}}}{{\sqrt {{x^2} - 9} }}dx = } \int {\frac{{{{\left( {3\sec \theta } \right)}^2}}}{{3\tan \theta }}} \left( {3\sec \theta \tan \theta } \right)d\theta \cr
& = \int {9{{\sec }^3}\theta } d\theta \cr
& = 9\int {{{\sec }^3}\theta d\theta } \cr
& {\text{Use the formula on page }}519{\text{ example 3}} \cr
& \int {{{\sec }^3}x} dx = \frac{1}{2}\sec x\tan x + \frac{1}{2}\ln \left| {\sec x + \tan x} \right| + C \cr
& 9\int {{{\sec }^3}\theta d\theta } = \frac{9}{2}\sec \theta \tan \theta + \frac{9}{2}\ln \left| {\sec \theta + \tan \theta } \right| + C \cr
& {\text{From the triangle sec}}\theta = \frac{x}{3}{\text{ and tan}}\theta {\text{ = }}\frac{{\sqrt {{x^2} - 9} }}{3} \cr
& = \frac{9}{2}\left( {\frac{x}{3}} \right)\left( {\frac{{\sqrt {{x^2} - 9} }}{3}} \right) + \frac{9}{2}\ln \left| {\frac{x}{3} + \frac{{\sqrt {{x^2} - 9} }}{3}} \right| + C \cr
& = \frac{{x\sqrt {{x^2} - 9} }}{2} + \frac{9}{2}\ln \left| {\frac{{x + \sqrt {{x^2} - 9} }}{3}} \right| + C \cr
& {\text{Then,}} \cr
& \int_4^6 {\frac{{{x^2}}}{{\sqrt {{x^2} - 9} }}} dx = \left[ {\frac{{x\sqrt {{x^2} - 9} }}{2} + \frac{9}{2}\ln \left| {\frac{{x + \sqrt {{x^2} - 9} }}{3}} \right|} \right]_4^6 \cr
& {\text{Evaluating}} \cr
& = \left[ {\frac{{6\sqrt {27} }}{2} + \frac{9}{2}\ln \left| {\frac{{6 + \sqrt {27} }}{3}} \right|} \right] - \left[ {\frac{{4\sqrt 7 }}{2} + \frac{9}{2}\ln \left| {\frac{{4 + \sqrt 7 }}{3}} \right|} \right] \cr
& {\text{Simplifying}} \cr
& = 3\sqrt {27} + \frac{9}{2}\ln \left( {6 + \sqrt {27} } \right) - 2\sqrt 7 - \frac{9}{2}\ln \left( {4 + \sqrt 7 } \right) \cr
& \approx 12.644 \cr} $$