Why does Helmer say it is impossible for Krogstad to remain in the bank?

Nora asks Torvald to reconsider dismissing Krogstad from the bank, but Torvald becomes angry, revealing that it is Krogstad’s post that he has promised to Mrs. Linde. Torvald says that he has already let Krogstad go and being talked into changing his mind by his wife would be embarrassing.