Answer
$-\dfrac{1712 \pi}{15}$
Work Step by Step
The flux through a surface can be defined only when the surface is orientable.
We know that $\iint_S F \cdot dS=\iint_S F \cdot n dS$
Here, $n$ denotes the unit vector.
Since, $\iint_S F \cdot n dS=\iint_D \sqrt {x^2+y^2}+z^3 dA$
Plug $z=x^2+y^2$ in the above equation.
$=\iint_D \sqrt {x^2+y^2}+(x^2+y^2) \sqrt {x^2+y^2}dA $
or, $= \int_{1}^{3} \int_0^{2 \pi} (r^2+1) \times \sqrt {r^2} \times r d\theta dr$
or, $= \int_0^{2 \pi} d\theta \times \int_{1}^{3} (r^4+r^2) dr$
or, $= (2 \pi) \times \int_{1}^{3}(r^4+r^2) dr$
Hence, the flux is $\iint_S F \cdot n dS=2 \pi [\dfrac{r^5}{5}+\dfrac{r^3}{3}]_1^{3}=-\dfrac{1712 \pi}{15}$
