How did Ben Gunn know that Jim's friend were in the stockade?

Gunn pointed out to that the British flag was flying in the blockade, this had to be Jim's friends because the pirates would never fly a British flag, but instead a Jolly Roger.