Answer
(a) Hyperbola. (b) $\phi=\frac{\pi}{6}$. $Y^2-X^2=1$ (c) See graph.
Work Step by Step
(a) Rewrite the original equation as $x^2+2\sqrt 3xy-y^2+0x+0y+2=0$ and we have $A=1, B=2\sqrt 3, C=-1, D=E=0,F=2$. The discriminant is $B^2-4AC=12+4=16\gt0$, thus the graph of the equation is a hyperbola.
(b) To eliminate the xy-term, we need a rotation of axes with an angle $\phi$ where $cot2\phi=\frac{A-C}{B}=\frac{1+1}{2\sqrt 3}=\frac{\sqrt 3}{3}$ which gives $2\phi=\frac{\pi}{3}$ and $\phi=\frac{\pi}{6}$.
The transformation formula gives: $x=X\cdot cos\frac{\pi}{6} - Y\cdot sin\frac{\pi}{6}=\frac{\sqrt 3}{2}X-\frac{1}{2}Y$, $y=X\cdot sin\frac{\pi}{6}+Y\cdot cos\frac{\pi}{6}=\frac{1}{2}X+\frac{\sqrt 3}{2}Y$ Use them in the original equation to get $(\frac{\sqrt 3}{2}X-\frac{1}{2}Y)^2+2\sqrt 3(\frac{\sqrt 3}{2}X-\frac{1}{2}Y)(\frac{1}{2}X+\frac{\sqrt 3}{2}Y)-(\frac{1}{2}X-\frac{\sqrt 3}{2}Y)^2+2=0$
Simplify this equation by multiplying 4 and use a formula $a^2-b^2=(a+b)(a-b)$ to get $Y^2-X^2=1$
(c) See graph.
