What is the "powerful emotion" that darkens the stranger's face when he looks at Hester?

I'm going to say anger darkened his face, although I don't know why he'd be angry. If he loved Hester, he'd have shown up long before and claimed his wife. I feel the same way about jealousy.... it just doesn't fit. I might even go with humiliation except that no one knew the relationship between the two. Thus, I'm stuck with anger.