why should he still feel the need for vengeance

Chillingworth wanted whatever revenge he could muster against the man with whom his wife committed adultery. Hester, of course had been punished, marked, and shunned, but none of those punishments were met out by the man who was her husband. Roger hadn't appeared until after the fact; Hester hadn't seen her husband, but upon his return, he chose to stay a mystery because he wanted to punish them both.