Answer
$$\dfrac{\pi h a^4}{10}$$
Work Step by Step
Our aim is to integrate the integral as follows:
$$ \int^{2\pi}_0 \int^a_0 \int^h _{\dfrac{h}{a}r}(x^2+y^2) dz (r dr) \space d \theta =\int ^{2\pi}_0 \int^a_0 \int^h _{\dfrac{hr}{a}} \space r^3 \space dz \space dr \space d\theta \\=\int^{2\pi}_0 \int^{a}_0 (hr^3-\frac{hr^4}{a}) \space dr \space d\theta \\=\int^{2\pi}_0 h[\dfrac{r^4}{4}-\frac{r^5}{5a}]^a_0 d\theta \\=\dfrac{\pi h a^4}{10}$$
