Why does Holden watch the game from the hill?

Holden watches from the hill because he doesn't want to go down and face the students sitting in the stands. He had forgotten the fencing equipment on the train, and the team hadn't been able to compete the previous day. He's frightened the other students will ignore him, so he keeps himself out of that position and stays away.