Answer
$\displaystyle \int x^2\sin2xdx=\displaystyle \frac{1}{2}{x}\sin {2x}+\frac{1}{4}\cos {2x}-\frac{1}{2}{x}^2\cos {2x}+C$
This answer is reasonable because $f(x)=F'(x)$
See image below
Work Step by Step
$I=\displaystyle \int x^2\sin2xdx$
$\displaystyle \left[\begin{array}{ll}
a=2x & x^2=(\frac{a}{2})^2\\
& \\
\frac{da}{dx}=2 & \frac{da}{2}=dx
\end{array}\right]$ Integration by substitution
$I=\displaystyle \int {(\frac{a}{2})}^2\sin a\frac{da}{2}\\
I=\displaystyle \frac{1}{8}\int {a}^2\sin ada$
$\displaystyle \left[\begin{array}{ll} u=a^2 & dv=\sin a \\ & \\ du=2a & v=-\cos a \end{array}\right]$ Integration by parts
$I=\displaystyle \frac{1}{8}(-a^2\cos a-\int -2a\cos ada)\\
I=\displaystyle \frac{1}{8}(-a^2\cos a+2\int a\cos ada)$
$\displaystyle \int a\cos ada$
$\displaystyle \left[\begin{array}{ll} u=2a & dv=\cos a \\ & \\ du=2 & v=\sin a \end{array}\right]$ Integration by parts
$I=\displaystyle -\frac{1}{8}a^2\cos a+\frac{1}{8}(2a\sin a-2\int \sin ada)\\
I=\displaystyle -\frac{1}{8}a^2\cos a+\frac{1}{4}a\sin a+\frac{1}{4}\cos a+C\\
I=\displaystyle -\frac{1}{8}{(2x)}^2\cos {2x}+\frac{1}{4}{2x}\sin {2x}+\frac{1}{4}\cos {2x}+C\\
I=\displaystyle -\frac{1}{2}{x}^2\cos {2x}+\frac{1}{2}{x}\sin {2x}+\frac{1}{4}\cos {2x}+C\\
I=\displaystyle \frac{1}{2}{x}\sin {2x}+\frac{1}{4}\cos {2x}-\frac{1}{2}{x}^2\cos {2x}+C$