Why does Tom refuse to sleep in the cave?

Tom is really looking for a shred of dignity after leaving prison. When Muley leads Tom and Casy to a cave, Tom refuses to hide saying, "I ain't gonna sleep in no cave." It is beneath him to sleep in a hole his brother and he dug looking for gold.