how do the men decide to determine hester's fitness as a mother

It is actually Dimmesdale's impassioned speech, which convinces the men that Hester is fit enough to be a mother. Dimmesdale argues that Pearl was sent by God to serve as a reminder of Hester's sin and keep her from sinning again.