Answer
(a) $(X-5)^2-Y^2=1$
(b) XY-coordinates, center $C(5,0)$, vertices $V_1(4,0)$, $V_2(6,0)$, foci $F(5\pm\sqrt 2,0)$.
xy-coordiantes: center $C(4,3)$, vertices $V_1(\frac{16}{5}, \frac{12}{5})$, $V_2(\frac{24}{5}, \frac{18}{5})$, foci $F_1(4+\frac{4\sqrt 2}{5}, 3+\frac{3\sqrt 2}{5})$, $F_2(4-\frac{4\sqrt 2}{5}, 3-\frac{3\sqrt 2}{5})$
(c) $Y=\pm(X-5)$
$7x-y-25=0$, $x+7y-25=0$
Work Step by Step
(a) The original equation gives $A=7, B=48, C=-7$. To remove the xy-term, we need to rotate the axes with an angle $\phi$ which is given by $\cot2\phi=\frac{A-C}{B}=\frac{14}{48}=\frac{7}{24}$. With $tan2\phi=\frac{24}{7}$, we have $cos2\phi=\frac{7}{25}$ and $sin\phi=\sqrt {\frac{1-cos2\phi}{2}}=\frac{3}{5}$, $cos\phi=\sqrt {\frac{1+cos2\phi}{2}}=\frac{4}{5}$. Use the rotation formula:
$x=Xcos\phi-Ysin\phi=\frac{4}{5}X-\frac{3}{5}Y$, $y=Xsin\phi+Ycos\phi=\frac{3}{5}X+\frac{4}{5}Y$.
Replace the variable in the original equation to get:
$7(\frac{4}{5}X-\frac{3}{5}Y)^2+48(\frac{4}{5}X-\frac{3}{5}Y)(\frac{3}{5}X+\frac{4}{5}Y)-7(\frac{3}{5}X+\frac{4}{5}Y)^2-200(\frac{4}{5}X-\frac{3}{5}Y)-150(\frac{3}{5}X+\frac{4}{5}Y)+600=0$
Multiply 25 on both sides and combine the like terms to get $625X^2-625Y^2-6250X+15000=0$ or $X^2-Y^2-10X+24=0$ which gives $(X-5)^2-Y^2=1$ that is a hyperbola.
(b) In the XY-coordinates, $a=1, b=1, c=\sqrt 2$, the center is at $C(5,0)$, vertices at $5\pm1,0$ or $V_1(4,0)$ and $V_2(6,0)$, and foci at $(5\pm\sqrt 2,0)$.
To find corresponding values in the xy-system, use the conversion formula given in (a):
$x=Xcos\phi-Ysin\phi=\frac{4}{5}X-\frac{3}{5}Y$, $y=Xsin\phi+Ycos\phi=\frac{3}{5}X+\frac{4}{5}Y$.
The center is at $(\frac{4}{5}\times5,\frac{3}{5}\times5)$ or $(4,3)$
The vertices are at $(\frac{4}{5}\times4, \frac{3}{5}\times4)$ or $V_1(\frac{16}{5}, \frac{12}{5})$, and $(\frac{4}{5}\times6, \frac{3}{5}\times6)$ or $V_2(\frac{24}{5}, \frac{18}{5})$
The foci are at $(\frac{4}{5}((5\pm\sqrt 2), \frac{3}{5}((5\pm\sqrt 2))$ or $F_1(4+\frac{4\sqrt 2}{5}, 3+\frac{3\sqrt 2}{5})$ and $F_2(4-\frac{4\sqrt 2}{5}, 3-\frac{3\sqrt 2}{5})$
(c) The equations of the asymptotes in XY-coordinates are $Y=\pm(X-5)$ that is $Y=X-5$ and $Y=-X+5$
To find the asymptotes in the xy-system, use the conversion formula:
$X=x\cdot cos\phi+y\cdot sin\phi=\frac{4}{5}x+\frac{3}{5}y$ and $Y=-x\cdot sin\phi+y\cdot cos\phi=-\frac{3}{5}x+\frac{4}{5}y$
and replace the $XY$ variables to get
asymptote-1: $-\frac{3}{5}x+\frac{4}{5}y=\frac{4}{5}x+\frac{3}{5}y-5$ which gives $7x-y-25=0$
asymptote-2: $-\frac{3}{5}x+\frac{4}{5}y=-\frac{4}{5}x-\frac{3}{5}y+5$ which gives $x+7y-25=0$