Why is it so difficult for Bigger to accept Mary Dalton's friendship?

Mary is a white girl and a the daughter and heiress of the Daltons. Both put her in a completely different social class than Bigger. Mary is a bit of a rebel both socially and politically which spurs her interest in Bigger but there is little about Mary that Bigger can relate to.