Answer
$$ - \frac{{{{\cot }^4}3x}}{{12}} + \frac{{{{\cot }^2}3x}}{6} + \frac{1}{3}\ln \left| {\sin 3x} \right| + C$$
Work Step by Step
$$\eqalign{ & \int {{{\cot }^5}3x} dx \cr & {\text{write as}} \cr & = \int {{{\cot }^3}3x} {\cot ^2}3xdx \cr & {\text{Use the pythagorean identity }}{\cot ^2}\theta = {\csc ^2}\theta - 1 \cr & = \int {{{\cot }^3}3x} \left( {{{\csc }^2}3x - 1} \right)dx \cr & {\text{multiply}} \cr & = \int {\left( {{{\cot }^3}3x{{\csc }^2}3x - {{\cot }^3}3x} \right)} dx \cr & = \int {{{\cot }^3}3x{{\csc }^2}3x} dx - \int {{{\cot }^3}3x} dx \cr & {\text{rewrite}} \cr & = - \frac{1}{3}\int {{{\cot }^3}3x\left( { - 3{{\csc }^2}3x} \right)} dx - \int {\cot 3x{{\cot }^2}3x} dx \cr & = - \frac{1}{3}\int {{{\cot }^3}3x\left( { - 3{{\csc }^2}3x} \right)} dx - \int {\cot 3x\left( {{{\csc }^2}3x - 1} \right)} dx \cr & = - \frac{1}{3}\int {{{\cot }^3}3x\left( { - 3{{\csc }^2}3x} \right)} dx - \int {\cot 3x{{\csc }^2}3x} dx + \int {\cot 3x} dx \cr & = - \frac{1}{3}\int {{{\cot }^3}3x\left( { - 3{{\csc }^2}3x} \right)} dx + \frac{1}{3}\int {\cot 3x\left( { - 3{{\csc }^2}3x} \right)} dx + \frac{1}{3}\int {\frac{{3\cos 3x}}{{\sin 3x}}} dx \cr & \cr & {\text{integrate}} \cr & {\text{ = }} - \frac{1}{3}\left( {\frac{{{{\cot }^4}3x}}{4}} \right) + \frac{1}{3}\left( {\frac{{{{\cot }^2}3x}}{2}} \right) + \frac{1}{3}\ln \left| {\sin 3x} \right| + C \cr & {\text{ = }} - \frac{{{{\cot }^4}3x}}{{12}} + \frac{{{{\cot }^2}3x}}{6} + \frac{1}{3}\ln \left| {\sin 3x} \right| + C \cr} $$