Answer
$-6$
Work Step by Step
Apply formula (2),
$\displaystyle \int_{0}^{1}\int_{1}^{2}(4x^{3}-9x^{2}y^{2})dy dx=\displaystyle \int_{0}^{1}[\int_{1}^{2}(4x^{3}-9x^{2}y^{2})dy] dx$
...treat $x$ as a constant in the inner integral.
$=\displaystyle \int_{0}^{1}[4x^{3}y-9x^{2}\frac{y^{3}}{3}]_{y=1}^{y=2}\ \ dx$
$=\displaystyle \int_{0}^{1}[4x^{3}y-3x^{2}y^{3}]_{y=1}^{y=2}\ \ dx$
$=\displaystyle \int_{0}^{1}[(4x^{3}\cdot 2-3x^{2}\cdot 2^{3})-(4x^{3}\cdot 1-3x^{2}\cdot 1^{3})]\ \ dx$
$=\displaystyle \int_{0}^{1}[8x^{3}-24x^{2}-4x^{3}+3x^{2}]dx$
$=\displaystyle \int_{0}^{1}(4x^{3}-21x^{2})dx$
$=[4\displaystyle \cdot\frac{x^{4}}{4}-21\cdot\frac{x^{3}}{3}]_{0}^{1}$
$=[x^{4}-7x^{3}]_{0}^{1}$
$=(1-7)-(0-0)$
$=-6$
