Answer
$x^2y^2-6xy+9$
Work Step by Step
Using $(a\pm b)^2=a^2\pm2ab+b^2$ or the square of a binomial, the square of the given expression, $
(xy-3)^2
,$ is
\begin{array}{l}\require{cancel}
(xy)^2-2(xy)(3)+(3)^2
\\\\=
x^2y^2-6xy+9
.\end{array}