Why is Ralph frustrated with the others?

Ralph and Simon are the only ones building the shelters. Ralph is frustrated because rules were made and agreed to, including the building of shelters, but most of the boys have forgotten them. They simply swim, eat, play and sleep. There is no accountability.