Answer
$$\int\sqrt{\frac{x^3-3}{x^{11}}}dx=\frac{2}{27}\Big(1-\frac{3}{x^3}\Big)^{3/2}+C$$
Work Step by Step
$$A=\int\sqrt{\frac{x^3-3}{x^{11}}}dx=\int\sqrt{\frac{1}{x^8}\times\frac{x^3-3}{x^3}}dx$$ $$A=\int\frac{1}{x^4}\sqrt{\frac{x^3-3}{x^3}}dx=\int\frac{1}{x^4}\sqrt{\Big(1-\frac{3}{x^3}\Big)}dx$$
We set $u=1-\frac{3}{x^3}$
Then $$du=0-\Big(\frac{(3)'x^3-3(x^3)'}{x^6}\Big)dx=-\Big(\frac{-9x^2}{x^6}\Big)dx=\frac{9}{x^4}dx$$
That means, $$\frac{1}{x^4}dx=\frac{1}{9}du$$
Therefore, $$A=\frac{1}{9}\int\sqrt udu=\frac{1}{9}\int u^{1/2}du$$ $$A=\frac{1}{9}\times\frac{u^{3/2}}{\frac{3}{2}}+C$$ $$A=\frac{u^{3/2}}{\frac{27}{2}}+C=\frac{2u^{3/2}}{27}+C$$ $$A=\frac{2}{27}\Big(1-\frac{3}{x^3}\Big)^{3/2}+C$$