Answer
$$\frac{\pi }{{24}}$$
Work Step by Step
$$\eqalign{
& \int_0^{\pi /6} {\int_0^{\cos 3\theta } r } drd\theta \cr
& = \int_0^{\pi /6} {\left[ {\int_0^{\cos 3\theta } r dr} \right]} d\theta \cr
& {\text{solve the inner integral and treat }}\theta {\text{ as a constant}} \cr
& = \int_0^{\cos 3\theta } r dr \cr
& {\text{then}} \cr
& = \left[ {\frac{{{r^2}}}{2}} \right]_0^{\cos 3\theta } \cr
& {\text{evaluating the limits in the variable }}r \cr
& = \frac{{{{\left( {\cos 3\theta } \right)}^2}}}{2} - \frac{{{{\left( 0 \right)}^2}}}{2} \cr
& = \frac{{{{\cos }^2}3\theta }}{2} \cr
& {\text{simplify by using trigonometric identities}} \cr
& = \frac{1}{2}\left( {\frac{{1 + \cos 6\theta }}{2}} \right) \cr
& = \frac{{1 + \cos 6\theta }}{4} \cr
& {\text{then}} \cr
& \int_0^{\pi /6} {\left[ {\int_0^{\cos 3\theta } r dr} \right]} d\theta = \int_0^{\pi /6} {\left( {\frac{{1 + \cos 6\theta }}{4}} \right)} d\theta \cr
& = \frac{1}{4}\int_0^{\pi /6} {\left( {1 + \cos 6\theta } \right)} d\theta \cr
& {\text{integrating }} \cr
& = \frac{1}{4}\left( {\theta + \frac{1}{6}\sin 6\theta } \right)_0^{\pi /6} \cr
& {\text{evaluate}} \cr
& = \frac{1}{4}\left( {\frac{\pi }{6} + \frac{1}{6}\sin 6\left( {\frac{\pi }{6}} \right)} \right) - \frac{1}{4}\left( {0 + \frac{1}{6}\sin 6\left( 0 \right)} \right) \cr
& = \frac{1}{4}\left( {\frac{\pi }{6} + 0} \right) - \frac{1}{4}\left( 0 \right) \cr
& = \frac{\pi }{{24}} \cr} $$
