Answer
(a) $\langle -2,17,-5 \rangle$
(b) $\langle -\frac{\sqrt {318}}{159},\frac{17\sqrt {318}}{318},-\frac{5\sqrt {318}}{318} \rangle$
Work Step by Step
(a) Given $\vec u=\langle 1,1,3 \rangle$ and $\vec v=\langle 5,0,-2 \rangle$, by definition the cross product is given by $\vec u\times\vec v=\langle (1\times(-2)-3\times0),(3\times5-1\times(-2)),(1\times0-1\times5) \rangle=\langle -2,17,-5 \rangle$
(b) Since the direction of the cross product is perpendicular to both $\vec u$ and $\vec v$, we can find a unit vector perpendicular to both $\vec u$ and $\vec v$ as $\vec n=\frac{1}{\sqrt {(-2)^2+(17)^2+(-5)^2}}\langle -2,17,-5 \rangle=\langle -\frac{\sqrt {318}}{159},\frac{17\sqrt {318}}{318},-\frac{5\sqrt {318}}{318} \rangle$