Answer
$y_p(x)=(Ax+B)e^{x} \cos x +(Cx+D)e^{x} \sin x$
Work Step by Step
We are given that $y''-y'-2y=xe^x \cos x$
Consider $G(x)=e^{\alpha x} A(x) \sin mx $ or $G(x)=e^{\beta x} A(x) \cos mx $
The trial solution for the method of undetermined coefficients is defined as:
$y_p(x)=e^{\alpha x} B(x) \sin mx +e^{\beta x} C(x) \cos mx$
On comparing with the given equation we get $m=k=1$ .
and the degree of the polynomials $B(x) ; C(x)=1$
Thus, the trial solution for the method of undetermined coefficients is:
$y_p(x)=(Ax+B)e^{x} \cos x +(Cx+D)e^{x} \sin x$
