Does the speaker find it depressing that such impoverished people exist? Or does he find it depressing to see such people? Do you think the speaker's sympathies are with the suffering lower class? Or with the poor rich class that has to look at them every

The speaker is bitter over the fact that such people exist because the reason for their existence is the horrible treatment the Irish have received from the British and the British landlords.  His satire is so exaggerated that one must understand that he is completely opposed to the rich and the way they treat those who are poor.  Swift has no love for the rich British.