Answer
$$ = \frac{1}{4}{\sec ^3}x\tan x - \frac{1}{8}\sec x\tan x - \frac{1}{8}{\text{ln}}\left| {\sec x + \tan x} \right| + C$$
Work Step by Step
$$\eqalign{
& \int {{{\tan }^2}x{{\sec }^3}x} dx \cr
& {\text{Use the pythagorean identity}} \cr
& = \int {\left( {{{\sec }^2}x - 1} \right){{\sec }^3}x} dx \cr
& = \int {\left( {{{\sec }^5}x - {{\sec }^3}x} \right)} dx \cr
& = \int {{{\sec }^5}x} dx - \int {{{\sec }^3}x} dx{\text{ }}\left( {\bf{1}} \right) \cr
& = \int {{{\sec }^3}x{{\sec }^2}x} dx - \int {{{\sec }^3}x} dx \cr
& {\text{Integrate }}\int {{{\sec }^5}x} dx{\text{ by parts, }} \cr
& u = {\sec ^3}x \Rightarrow du = 3{\sec ^3}x\tan x \cr
& dv = {\sec ^2}x \Rightarrow v = \tan x \cr
& \int {{{\sec }^5}x} dx = {\sec ^3}x\tan x - 3\int {\tan x} {\sec ^3}x\tan xdx \cr
& \int {{{\sec }^5}x} dx = {\sec ^3}x\tan x - 3\int {{{\tan }^2}x} {\sec ^3}xdx \cr
& {\text{Use the pythagorean identity}} \cr
& \int {{{\sec }^5}x} dx = {\sec ^3}x\tan x - 3\int {\left( {{{\sec }^2}x - 1} \right)} {\sec ^3}xdx \cr
& \int {{{\sec }^5}x} dx = {\sec ^3}x\tan x - 3\int {\left( {{{\sec }^5}x - {{\sec }^3}x} \right)} dx \cr
& \int {{{\sec }^5}x} dx = {\sec ^3}x\tan x - 3\int {{{\sec }^5}x} dx + 3\int {{{\sec }^3}x} dx \cr
& 4\int {{{\sec }^5}x} dx = {\sec ^3}x\tan x + 3\int {{{\sec }^3}x} dx \cr
& \int {{{\sec }^5}x} dx = \frac{1}{4}{\sec ^3}x\tan x + \frac{3}{4}\int {{{\sec }^3}x} dx \cr
& {\text{Substitute this result in }}\left( {\bf{1}} \right) \cr
& = \frac{1}{4}{\sec ^3}x\tan x + \frac{3}{4}\int {{{\sec }^3}x} dx - \int {{{\sec }^3}x} dx \cr
& = \frac{1}{4}{\sec ^3}x\tan x - \frac{1}{4}\int {{{\sec }^3}x} dx \cr
& {\text{Recall that }}\int {{{\sec }^3}x} dx = \frac{1}{2}\sec x\tan x + \frac{1}{2}{\text{ln}}\left| {\sec x + \tan x} \right|{\text{ + C}} \cr
& {\text{Then,}} \cr
& = \frac{1}{4}{\sec ^3}x\tan x - \frac{1}{8}\sec x\tan x - \frac{1}{8}{\text{ln}}\left| {\sec x + \tan x} \right| + C \cr} $$
