Who does Dimmesdale referring to when he says there is someone whose sin is worse than his or Hester's

Chillingworth's sins are far worse. Dimmesdale has begun to understand just how deep the evil in this man runs. He understands that Chillingworth means to ruin them all.