Why is Higgins unconcerned about what will happen to Eliza? Why does this seem to hurt her feelings? (Act 4)

In this pivotal act, the relationship between Eliza and Higgins finally explodes. Although Higgins pretends to be unconcerned with Eliza's fate, it is revealed that there has been a deeper feeling between them. Eliza feels that, although the situation was twisted, there was some kind of bond between them.