Answer
$\dfrac{4(-2+4\sqrt {2}+9\sqrt 3)}{105}$
Work Step by Step
Since, $\iint_S F \cdot dS=\iint_S F \cdot n dS$
where, $n$ denotes the unit vector.
and $dS=n dA=\pm \dfrac{(r_u \times r_v)}{|r_u \times r_v|}|r_u \times r_v| dA=\pm (r_u \times r_v) dA$
The flux through a surface can be defined only when the surface is orientable.
Here,we have $\iint_S y dS =\iiint_{D}y \sqrt {(x^{1/2})^2+(\sqrt y)^2+1^2} dA$
or, $= \iint_D y\sqrt {x+y+1} dA$
or, $=\int_{0}^{1} \int_{0}^{1}y \times \sqrt {x+y+1} dy dx$
and $\dfrac{2}{3} \times \int_{0}^{1}y(y+2)^{3/2}-y(y+1)^{3/2} dy=\dfrac{2}{3} \times \int_{0}^{1}y \times (y+2)^{3/2} dy -(2/3) \int_0^1 y(y+1)^{3/2} dy$
We need to plug in $y=m^2-2$ in the first integral and $y=n^2-1$ in the second integral.
We have: $\dfrac{2}{3} \times \int_{\sqrt 2}^{\sqrt 3} (m^2-2) m^3 (2m) dm -(2/3) \int_{1}^{\sqrt 2} (n^2-1) n^3 (2n) dn=\dfrac{4}{3} \int_{\sqrt 2}^{\sqrt 3} m^6-2m^4 dt- (4/3) \int_{1}^{\sqrt 2} (n^6-n^4) db$
Hence, we have $\iint_S y dS=\dfrac{4(-2+4\sqrt {2}+9\sqrt 3)}{105}$
