Answer
$\kappa=\dfrac{|ab \omega^3|}{[a^2 \omega^2 \sin^2 \omega t+b^2 \omega^2 \cos^2 \omega t]^{3/2}}$
Work Step by Step
Given: $x=a \cos \omega t, y=b \sin \omega t$
$x'=-a \omega \sin \omega t, y'=b \omega \cos \omega t $
and $x''=-a \omega^2 \cos \omega t, y''=-b \omega^2 \sin \omega t $
Use formula $\kappa=\dfrac{|\dot{x} \ddot{y}-\dot{y} \ddot{x}|}{[\dot{x}^2+\dot{y}^2]^{3/2}}$
$\kappa=\dfrac{|(-a \omega \sin \omega t)(-b \omega^2 \sin \omega t)-(b \omega \cos \omega t)(-a \omega^2 \cos \omega t)|}{[(-a \omega^2 \sin \omega t)^2+(b \omega \cos \omega t)^2]^{3/2}}$
$\kappa=\dfrac{|ab \omega^3|}{[a^2 \omega^2 \sin^2 \omega t+b^2 \omega^2 \cos^2 \omega t]^{3/2}}$