why is the narrator sometimes angry with and disappointed in Doodle

Doodle cannot complete the physical feats that the narrator wants Doodle to perform. The narrator cannot stand the idea of having a brother who is not "all there". He also gets tired of Doodle's incessant questions.