Why are the children fearful when they hear about the "riders?"

These are the "night men" who go around lynching black people at night.

They hear that the "devilish night men" have been "set off." Cassie worries that the men are coming after them because of what they did to the bus, and Stacey feels as if it were his fault because it was his idea to dig the ditch.