Answer
$y(x)=c_1+c_2e^{3x}\sin 4x+c_3e^{3x}\cos 4x$
Work Step by Step
We are given $y'''-6y''+25y'=x^2$
Solve the auxiliary equation for the differential equation.
$r^3-6r^3+25r=0$
Factor and solve for the roots.
$r(r^2-6r+25)=0$
Roots are: $r_1=0$, as a multiplicity of 1, $r_2=3+4i$ as a multiplicity of 1 and $r_3=3-4i$ as a multiplicity of 1.
The general solution is
$y(x)=c_1+c_2e^{3x}\sin 4x+c_3e^{3x}\cos 4x+A_0x+B_0x^2$
The trial solution for $y_p=A_0x+B_0x^2$ can be computed as by plugging back into the given differential equation.
$0-12B_0+25A_0+50B_0x=x^2$
then
$A_0=0\\
B_=0$
Hence,
$y(x)=c_1+c_2e^{3x}\sin 4x+c_3e^{3x}\cos 4x$
