Answer
(a) parabola.
(b) $\phi\approx36.9^{\circ}$,
$5Y^2-12X+28Y+20=0$
(c) See graph.
Work Step by Step
(a) Rewrite the original equation as $9x^2-24xy+16y^2-100x+100y+100=0$ and we have $A=9, B=-24, C=16, D=-100,E=100,F=100$. The discriminant is $B^2-4AC=24^2-4\times9\times16=0$, thus the graph of the equation is a parabola.
(b) To eliminate the xy-term, we need a rotation of axes with an angle $\phi$ where $cot2\phi=\frac{A-C}{B}=\frac{9-16}{-24}=\frac{7}{24}$ which gives $2\phi\approx73.74^{\circ}$ and $\phi\approx36.9^{\circ}$.
With $tan2\phi=\frac{24}{7}$, we have $cos2\phi=\frac{7}{25}$ and $sin\phi=\sqrt {\frac{1-cos2\phi}{2}}=\frac{3}{5}$ and $cos\phi=\sqrt {\frac{1+cos2\phi}{2}}=\frac{4}{5}$
The transformation formula gives:
$x=X\cdot \frac{4}{5} - Y\cdot \frac{3}{5}=\frac{4}{5}X-\frac{3}{5}Y$,
$y=X\cdot \frac{3}{5}+Y\cdot\frac{4}{5}=\frac{3}{5}X+\frac{4}{5}Y$
Use them in the original equation to get $9(\frac{4}{5}X-\frac{3}{5}Y)^2-24(\frac{4}{5}X-\frac{3}{5}Y)(\frac{3}{5}X+\frac{4}{5}Y)+16(\frac{3}{5}X+\frac{4}{5}Y)^2-100(\frac{4}{5}X-\frac{3}{5}Y)+100(\frac{3}{5}X+\frac{4}{5}Y)+100=0$
Simplify this equation by multiplying 25 and combine like terms to get $5Y^2-12X+28Y+20=0$
(c) See graph.