Answer
$x_1(t)=-c_1e^{t}\cos3t+c_2e^{t}\sin3t$
$x_2(t)=c_1e^{t}\cos3t+c_2e^{t}\sin3t$
Work Step by Step
$x_1'=x_1 - 3x_2$
$x_2'=3x_1 + x_2$
We begin by rewriting the system in operator form as $(D−1)x_1 +3x_2=0$ (1)
$-3x_1+(D -1)x_2 =0 $ (2)
To eliminate $x_2$ between these two equations, we first operate on equation (2) with $\frac{1}{3}(D-1)$ to obtain:
$-(D-1)x_1+\frac{1}{3}(D-1)(D -1)x_2 =0$
Adding equation (1) to this equation eliminates $x_2$ and yields
$\frac{1}{3}(D^2-2D+10)x_2 =0 $
This constant coefficient differential equation has auxiliary polynomial
$P(r)=\frac{1}{3}(r^2-2r+10)=r^2-2r+10 \rightarrow r_1=1-3i; r_2=1+3i$
Consequently, $x_2(t)=c_1e^{t}\cos3t+c_2e^{t}\sin3t$
From equation $-3x_1+(D -1)x_2 =0 $
$x_1(t)=-c_1e^{t}\cos3t+c_2e^{t}\sin3t$