Answer
$$\frac{{\sin {x^2}}}{2} - \frac{{{{\sin }^3}{x^2}}}{3} + \frac{{{{\sin }^5}{x^2}}}{{10}} + C$$
Work Step by Step
$$\eqalign{
& \int x {\cos ^5}\left( {{x^2}} \right)dx \cr
& {\text{Let }}t = {x^2},{\text{ }}dt = 2xdx \cr
& \int x {\cos ^5}\left( {{x^2}} \right)dx = \frac{1}{2}\int {{{\cos }^5}} tdt \cr
& {\text{Rewrite the integrand}} \cr
& = \frac{1}{2}\int {{{\left( {{{\cos }^2}t} \right)}^2}\cos t} dt \cr
& = \frac{1}{2}\int {{{\left( {1 - {{\sin }^2}t} \right)}^2}\cos t} dt \cr
& {\text{Expand}} \cr
& = \frac{1}{2}\int {\left( {1 - 2{{\sin }^2}t + {{\sin }^4}t} \right)\cos t} dt \cr
& {\text{Let }}u = \sin t,{\text{ }}du = \cos tdt \cr
& = \frac{1}{2}\int {\left( {1 - 2{u^2} + {u^4}} \right)} du \cr
& {\text{Integrating}} \cr
& = \frac{1}{2}u - \frac{{{u^3}}}{3} + \frac{1}{2}\left( {\frac{{{u^5}}}{5}} \right) + C \cr
& = \frac{u}{2} - \frac{{{u^3}}}{3} + \frac{{{u^5}}}{{10}} + C \cr
& {\text{Back - substitute}} \cr
& = \frac{{\sin t}}{2} - \frac{{{{\sin }^3}t}}{3} + \frac{{{{\sin }^5}t}}{{10}} + C \cr
& = \frac{{\sin {x^2}}}{2} - \frac{{{{\sin }^3}{x^2}}}{3} + \frac{{{{\sin }^5}{x^2}}}{{10}} + C \cr} $$