why was jim angry with antonia?

I'm really not sure, I can only speculate. Jim took an awful beating and his romantic love for Antonia was never realized. He could also be angry that Antonia would work for a guy who tried to rape her, or perhaps had even raped her before.