Answer
$$\int_{0}^{\pi / 4} \int_{\sqrt{3}}^{\sqrt{3} \cos \theta} r d r d \theta =\frac{3}{8}(1-\frac{ \pi}{2} )$$
Work Step by Step
Given $$\int_{0}^{\pi / 4} \int_{\sqrt{3}}^{\sqrt{3} \cos \theta} r d r d \theta $$
So, we have
\begin{aligned}
I=\int_{0}^{\pi / 4} \int_{\sqrt{3}}^{\sqrt{3} \cos \theta} r d r d \theta &=\int_{0}^{\pi / 4}\left[\frac{r^{2}}{2}\right]_{\sqrt{3}}^{\sqrt{3} \cos \theta} d \theta\\
&=\int_{0}^{\pi / 4}\left(\frac{3 \cos ^{2} \theta}{2}-\frac{3}{2}\right) d \theta\\
\end{aligned}
as $$\cos ^{2} \theta=\frac{1}{2}(1+\cos 2 \theta)$$
So, we get
\begin{aligned}
I&=\int_{0}^{\pi / 4}\left(\frac{3}{4}(1+\cos 2 \theta)-\frac{3}{2}\right) d \theta \\ &=\int_{0}^{\pi / 4}\left(\frac{3}{4} \cos 2 \theta-\frac{3}{4}\right) d \theta\\
&=\left[\frac{3}{8} \sin 2 \theta-\frac{3}{4} \theta\right]_{0}^{\pi / 4}\\
&=\frac{3}{8} \sin \frac{\pi}{2}-\frac{3 \pi}{16} -(\frac{3}{8} \sin 0-0)\\
&=\frac{3}{8}-\frac{3 \pi}{16}
\end{aligned}
