Answer
$$\frac{\sqrt{3}}{9} w H^{3},\ \ \ \frac{\sqrt{3}}{3} \ell w H^{2}$$
where ($w=\rho g$).
Work Step by Step
Place the origin at the lower vertex of the trough and orient the positive $y$ -axis pointing upward. First, consider the faces at the front and back ends of the trough. A horizontal strip at height $y$ has a length of $\frac{2 y}{\sqrt{3}}$ and is at a depth of $H-y .$ Thus,
\begin{align*}
F&=w \int_{0}^{H}(H-y) \frac{2 y}{\sqrt{3}} d y\\
&=\left.w\left(\frac{H}{\sqrt{3}} y^{2}-\frac{2}{3 \sqrt{3}} y^{3}\right)\right|_{0} ^{H}\\
&=\frac{\sqrt{3}}{9} w H^{3}
\end{align*}
where ($w=\rho g$).
For the slanted sides, we note that each side makes an angle of $60^{\circ}$ with the horizontal. If we let $\ell$ denote the length of the trough, then
\begin{align*}
F&=\frac{2 w \ell}{\sqrt{3}} \int_{0}^{H}(H-y) d y\\
&=\frac{\sqrt{3}}{3} \ell w H^{2}
\end{align*}
