Answer
$i+(sint-tcost)j-(tsint+cost)k$
Yes, $a \times b$ is orthogonal to both $a$ and $b$.
Work Step by Step
$a=ti+cost j+sint k= \lt t,cost,sint \gt$
$b=i-sint j+cost k= \lt 1,-sint,cost \gt$
$a\times b= \begin{vmatrix} i&j&k \\ t&cost&sint\\1&-sint &cost\end{vmatrix}$
Expand along the first row:
$a \times b=i+(sint-tcost)j-(tsint+cost)k$
To verify that it is orthogonal to $a$, we will compute:
$(a\times b).a=t(1)+cost(sint-tcost)+sint(-tsint-cost)=0$
To verify that it is orthogonal to $b$, we will compute:
$(a\times b).b=1(1)-sint(sint-tcost)+cost(-tsint-cost)=0$
Yes, $a \times b$ is orthogonal to both $a$ and $b$.
