Answer
$$\sqrt 3 - \frac{\pi }{3}$$
Work Step by Step
$$\eqalign{
& \int_0^{\sqrt 3 /2} {\frac{{{x^2}}}{{{{\left( {1 - {x^2}} \right)}^{3/2}}}}} dx \cr
& {\text{The integrand contains the form }}{a^2} - {x^2} \cr
& 1 - {x^2} \to a = 1 \cr
& {\text{Use the change of variable }}x = a\sin \theta \cr
& x = \sin \theta ,\,\,\,dx = \cos \theta d\theta \cr
& \cr
& {\text{Substituting}} \cr
& \int {\frac{{{x^2}}}{{{{\left( {1 - {x^2}} \right)}^{3/2}}}}dx} = \int {\frac{{{{\sin }^2}\theta }}{{{{\left( {1 - {{\sin }^2}\theta } \right)}^{3/2}}}}\cos \theta d\theta } \cr
& = \int {\frac{{{{\sin }^2}\theta }}{{{{\left( {{{\cos }^2}\theta } \right)}^{3/2}}}}\cos \theta d\theta } \cr
& = \int {\frac{{{{\sin }^2}\theta }}{{{{\cos }^2}\theta }}d\theta } = \int {{{\tan }^2}\theta } d\theta \cr
& = \int {\left( {{{\sec }^2}\theta - 1} \right)} d\theta \cr
& = \tan \theta - \theta + C \cr
& {\text{Write in terms of }}x \cr
& = \frac{x}{{\sqrt {1 - {x^2}} }} - {\sin ^{ - 1}}x + C \cr
& {\text{Therefore,}} \cr
& \int_0^{\sqrt 3 /2} {\frac{{{x^2}}}{{{{\left( {1 - {x^2}} \right)}^{3/2}}}}} dx = \left[ {\frac{x}{{\sqrt {1 - {x^2}} }} - {{\sin }^{ - 1}}x} \right]_0^{\sqrt 3 /2} \cr
& = \left[ {\frac{{\sqrt 3 /2}}{{\sqrt {1 - {{\left( {\sqrt 3 /2} \right)}^2}} }} - {{\sin }^{ - 1}}\left( {\frac{{\sqrt 3 }}{2}} \right)} \right] - 0 \cr
& {\text{Simplifying}} \cr
& {\text{ = }}\sqrt 3 - \frac{\pi }{3} \cr} $$
