Who was the captain most afraid of?

Billy Bones seems to be most frightened of encountering "the seafaring man with one leg". He is, however, afraid of both of his visitors, Black Dog and the blind man, because he knows they may be in possession of the black spot.... the mark of death.