How is Lennie lonely in chapter 4

all of the men are off at the whorehouse spending their weeks' pay except for the feeble threesome of Crooks, Candy and Lennie. The setting is the "little shed that leaned off the wall of the barn" (73) that makes up Crooks' quarters. Lennie, however, is lonely and doesn't understand the unwritten code of racial segregation. He appears in Crooks' doorway hoping to talk with Crooks.