Answer
$x=\frac{\sqrt 5}{5}(x'-2y')$ and $y=\frac{\sqrt 5}{5}(2x'+y')$
Work Step by Step
1. Based on the given equation, we have $A=4, B=-4, C=1$, the rotation angle satisfies $cot(2\theta)=\frac{A-C}{B}=\frac{4-1}{-4}=-\frac{3}{4}$ (quadrant II),
2. Let $x=-3, y=4$, we have $r=5$, $cos(2\theta)=-\frac{3}{5}$, with $\theta$ in quadrant I, thus $sin\theta=\sqrt {\frac{1-cos(2\theta)}{2}}=\sqrt {\frac{1+\frac{3}{5}}{2}}=\frac{2\sqrt 5}{5}$ and
$cos\theta=\sqrt {\frac{1+cos(2\theta)}{2}}=\sqrt {\frac{1-\frac{3}{5}}{2}}=\frac{\sqrt 5}{5}$
3. The formulas for the rotation are $x=x'cos\theta-y'sin\theta=\frac{\sqrt 5}{5}x'-\frac{2\sqrt 5}{5}y'=\frac{\sqrt 5}{5}(x'-2y')$ and $y=x'sin\theta+y'cos\theta=\frac{2\sqrt 5}{5}x'+\frac{\sqrt 5}{5}y'=\frac{\sqrt 5}{5}(2x'+y')$
