Answer
$|a \times b|^2=|a|^2 |b|^2-(a \cdot b)^2$
Work Step by Step
$|a \times b|^2=( |a| |b||sin \theta|)^2= |a|^2 |b|^2sin ^2\theta$
As we know : $sin ^2 \theta =1 -cos ^2 \theta$
Thus,
$ |a|^2 |b|^2sin ^2\theta= |a|^2 |b|^2-|a|^2 |b|^2cos ^2\theta$
Remember that
$(a \cdot b)^2=(|a|^2 |b|^2cos\theta) ^2=|a|^2 |b|^2cos^2\theta$
Hence, $|a \times b|^2=|a|^2 |b|^2-(a \cdot b)^2$
