Answer
The solution is $(-t,2t,t)$
Work Step by Step
$Ax=0 $
$\begin{bmatrix}
1& 0&3\\
3& -1 & 7\\
3 &1 &8\\
1 & 1 &5\\
-1 & 1 & -1
\end{bmatrix}.\begin{bmatrix}
x_1\\
x_2\\
x_3\\
\end{bmatrix}=\begin{bmatrix}
0\\
0\\
0
\end{bmatrix} $
$\begin{bmatrix}
1& 0&3|0\\
3& -1 & 7|0\\
3 &1 &8|0\\
1 & 1 &5|0\\
-1 & 1 & -1|0
\end{bmatrix}\approx^1\begin{bmatrix}
1& 0&3|0\\
0& -1 & -2|0\\
0 &1 &2|0\\
0 & 1 &2|0\\
0& 1 & 2|0
\end{bmatrix} \approx^2\begin{bmatrix}
1& 0&3|0\\
0& 1 & 2|0\\
0 &1 &2|0\\
0 & 1 &2|0\\
0& 1 & 2|0
\end{bmatrix} \approx^3 \begin{bmatrix}
1& 0&3|0\\
0& 1 & -2|0\\
0 &0 &0|0\\
0 & 0 &0|0\\
0& 0 & 0|0
\end{bmatrix} $
This matrix is now in row-echelon form.
$1. A_{12}(-3),A_{13}(-2), A_{15}(1)$
$2.M_2(-1)$
$3.A_{23}(-3),A_{24}(-1), A_{25}(-1)$
The solution set is:
$x_1+x_3=0$
$x_2-2x^3=0$
$x_3=t$
There is one free variable, which we take to be $x_3=t$, where t can assume any complex value. Applying back substitution yields:
$x_1+t=0 \rightarrow x_1=-t$
$x_2-2t=0 \rightarrow x_2=2t$
$x_3=t$
Hence, the solution is $(-t,2t,t)$
