Answer
$\frac{1}{4}\left( {{{\sec }^4}x - 4{{\sec }^2}x - 4\ln \left( {\cos x} \right)} \right) + C$
Work Step by Step
$$\eqalign{
& \int {{{\tan }^5}x} dx \cr
& {\text{Integrate by using a CAS }}\left( {{\text{See image below}}} \right){\text{ we obtain:}} \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\sec }^4}x - 4{{\sec }^2}x - 4\ln \left( {\cos x} \right)} \right) + C \cr
& \cr
& {\text{Integrating by tables using the entry 75}} \cr
& \int {{{\tan }^n}u} du = \frac{1}{{n - 1}}{\tan ^{n - 1}}u - \int {{{\tan }^{n - 2}}udu} \cr
& {\text{Therefore}}{\text{, let }}n = 5,{\text{ }}u = x \cr
& \int {{{\tan }^5}x} dx = \frac{1}{{5 - 1}}{\tan ^{5 - 1}}x - \int {{{\tan }^{5 - 2}}xdx} \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\tan ^4}x - \int {{{\tan }^3}xdx} \cr
& {\text{Integrating }}\int {{{\tan }^3}xdx} {\text{ using the entry 69}} \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\tan ^4}x - \left( {\frac{1}{2}{{\tan }^2}x + \ln \left| {\cos x} \right|} \right) + C \cr
& {\text{Use the pythagorean identity ta}}{{\text{n}}^2}x = {\sec ^2}x - 1 \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\left( {{{\sec }^2}x - 1} \right)^2} - \left( {\frac{1}{2}\left( {{{\sec }^2}x - 1} \right) + \ln \left| {\cos x} \right|} \right) + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\left( {{{\sec }^2}x - 1} \right)^2} - \left( {\frac{1}{2}{{\sec }^2}x - \frac{1}{2} + \ln \left| {\cos x} \right|} \right) + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\left( {{{\sec }^2}x - 1} \right)^2} - \frac{1}{2}{\sec ^2}x + \frac{1}{2} - \ln \left| {\cos x} \right| + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}{\left( {{{\sec }^2}x - 1} \right)^2} - \frac{1}{2}{\sec ^2}x - \ln \left| {\cos x} \right| + \frac{1}{2} + C \cr
& {\text{Factoring out }}\frac{1}{4} \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\left( {{{\sec }^2}x - 1} \right)}^2} - 2{{\sec }^2}x - 4\ln \left| {\cos x} \right| + 2} \right) + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\sec }^4}x - 2{{\sec }^2}x + 1 - 2{{\sec }^2}x - 4\ln \left| {\cos x} \right| + 2} \right) + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\sec }^4}x - 4{{\sec }^2}x - 4\ln \left| {\cos x} \right| + 3} \right) + C \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\sec }^4}x - 4{{\sec }^2}x - 4\ln \left| {\cos x} \right|} \right) + \frac{3}{4} + C \cr
& {\text{Combining constants}}{\text{, and let }}\cos x > 0 \cr
& \int {{{\tan }^5}x} dx = \frac{1}{4}\left( {{{\sec }^4}x - 4{{\sec }^2}x - 4\ln \left( {\cos x} \right)} \right) + C \cr
& {\text{We obtain the same result}} \cr} $$
