What did Death find painful about his duties?

Death tells us that the most painful part of his job is seeing "the survivors," "the leftover humans," "the ones who are left behind, crumbling among the jigsaw puzzle of realization, despair, and surprise."