Answer
$ \dfrac{2 \sqrt 2-1}{3}$
Work Step by Step
Let us consider $I=\int_{0}^{\pi/2} \int_{0}^{\sin x} \cos x \sqrt {1+\cos^2 x} dy \ dx \\ =\int_{0}^{\pi/2} y \cos x [ \sqrt {1+\cos^2 x} ]_{0}^{\sin x} \cos x \ dx \\=\int_{0}^{\pi/2} \sin x \cos x \sqrt {1+\cos^2 x} \ dx$
Set $1+\cos^2 x=u$ and $du=-2 \cos x \sin x dx$
Now, $I=\int_{0}^{\pi/2} \sin x \cos x \sqrt {1+\cos^2 x} \ dx=-\dfrac{1}{2} \times \int_{2}^{1} u^{1/2} du$
or, $=\dfrac{1}{2} \int_{1}^{2} \sqrt u du$
or, $=[\dfrac{u^{3/2}}{3}]_1^2$
or, $= \dfrac{2 \sqrt 2-1}{3}$
