Answer
boom is 36 ft, height of mainsail is 100 ft
Work Step by Step
mainsail and jib sail combine for 3000 $ft^2$
mainsail is 60% of the area (1800 $ft^2$)
height of mainsail is 2*boom+28
$h = 2b+28$
$1800 = base * height*1/2$
$1800 = b*(2b+28)/2$
$1800 = (2b^2+28b)/2$
$1800 = b^2+14b$
$b^2+14b-1800=0$
$b^2+50b-36b-1800=0$
$b(b+50)-36(b+50)=0$
$(b+50)(b-36)=0$
$b+50=0$
$b=-50$ (invalid since we can't have a negative length)
$b-36=0$
$b=36$
$h=2b+28$
$h=2*36+28$
$h=72+28$
$h=100$