Why does Grant bring up "moral obligations" during the discussion with Jefferson?

Grant reprimands Jefferson for making Miss Emma cry. He might as well be kind to the woman who has done so much for him. Grant talks about "moral obligations" that Jefferson still has to his family even though he is in prison.