Answer
(a) hyperbola.
(b) $\phi=\frac{\pi}{3}$.
$(X-1)^2-3Y^2=1$
(c) See graph.
Work Step by Step
(a) Rewrite the original equation as $2\sqrt 3x^2-6xy+\sqrt 3x+3y=0$ and we have $A=2\sqrt 3, B=-6, C=0, D=\sqrt 3,E=3,F=0$. The discriminant is $B^2-4AC=35-0=36\gt0$, thus the graph of the equation is a hyperbola.
(b) To eliminate the xy-term, we need a rotation of axes with an angle $\phi$ where $cot2\phi=\frac{A-C}{B}=\frac{2\sqrt 3}{-6}=-\frac{\sqrt 3}{3}$ which gives $2\phi=\frac{2\pi}{3}$ and $\phi=\frac{\pi}{3}$. The transformation formula gives:
$x=X\cdot cos\frac{\pi}{3} - Y\cdot sin\frac{\pi}{3}=\frac{1}{2}X-\frac{\sqrt 3}{2}Y$,
$y=X\cdot sin\frac{\pi}{3}+Y\cdot cos\frac{\pi}{3}=\frac{\sqrt 3}{2}X+\frac{1}{2}Y$
Use them in the original equation to get $2\sqrt 3(\frac{1}{2}X-\frac{\sqrt 3}{2}Y)^2-6(\frac{1}{2}X-\frac{\sqrt 3}{2}Y)(\frac{\sqrt 3}{2}X+\frac{1}{2}Y)+\sqrt 3(\frac{1}{2}X-\frac{\sqrt 3}{2}Y)+3(\frac{\sqrt 3}{2}X+\frac{1}{2}Y)=0$
Simplify this equation by multiplying 2 and combine like terms to get $3Y^2-X^2+2X=0$ or $(X-1)^2-3Y^2=1$
(c) See graph.
