Answer
$\displaystyle \frac{32}{5}(\ln 2)^{2}-\frac{64}{25}\ln 2+\frac{62}{125}$
Work Step by Step
Integration by parts:
$\displaystyle \int udv=uv-\int vdu$
The idea is to choose a relatively easy $dv$ to integrate, and
a u whose $u'$ does not complicate matters (best case: makes thing simpler).
----
$\left[\begin{array}{ll}
u=(\ln x)^{2} & dv=x^{4}dx\\
& \\
du=2\frac{\ln x}{x}dx, & v=\frac{1}{5}x^{5}
\end{array}\right]$
$I=\displaystyle \int_{1}^{2}x^{4}(\ln x)^{2}dx=uv|_{1}^{2}-\int_{1}^{2}vdu$
$=[\displaystyle \frac{x^{5}}{5}(\ln x)^{2}]_{1}^{2}-\int_{1}^{2}\frac{1}{5}x^{5}\cdot 2\frac{\ln x}{x}dxdx$
$=(\displaystyle \frac{32}{5}(\ln 2)^{2}-0)-\frac{2}{5}\int_{1}^{2}x^{4}\ln xdx$.
$=\displaystyle \frac{32}{5}(\ln 2)^{2}-\frac{2}{5}\int_{1}^{2}x^{4}\ln xdx$.
By parts, again, $\left[\begin{array}{ll}
u=\ln x & dv=x^{4}dx\\
& \\
du=\frac{1}{x}dx & v=\frac{1}{5}x^{5}
\end{array}\right]$
$\displaystyle \int_{1}^{2}x^{4}\ln xdx=[\frac{x^{5}}{5}\ln x]_{1}^{2}-\int_{1}^{2}\frac{1}{5}x^{5}\cdot\frac{1}{x}dx==[\frac{x^{5}}{5}\ln x]_{1}^{2}-\frac{1}{5}\int_{1}^{2}x^{4}dx$
$=\displaystyle \frac{32}{5}\ln 2-0-\frac{1}{5}[\frac{x^{5}}{5}]_{1}^{2}$
$=\displaystyle \frac{32}{5}\ln 2-(\frac{32}{25}-\frac{1}{25})$
$=\displaystyle \frac{32}{5}\ln 2-\frac{31}{25}$
$I=\displaystyle \frac{32}{5}(\ln 2)^{2}-\frac{2}{5}(\frac{32}{5}\ln 2-\frac{31}{25})$
$=\displaystyle \frac{32}{5}(\ln 2)^{2}-\frac{64}{25}\ln 2+\frac{62}{125}$.