Chapter 8 - Linear Differential Equations of Order n - 8.3 The Method of Undetermined Coefficients: Annihilators - Problems - Page 526: 46

See below

We can notice that: $F(x)=\sin^2 x$ Obtain: $(D+3)(D-1)y_p(x)=\sin^2 x$ Therefore, the general solution for the given differential equation is: $y(x)=c_1e^{x}+c_2e^{-3x}\sin x+A_0+A_1\cos 2x +A_2\sin 2x$ The trial solution is $y_p=A_0+A_1\cos 2x +A_2\sin 2x$.