What was it about the radleys that made the town scared of them?

The Radley's kept to themselves. They also did the socially unthinkable taboo of not attending church. Their doors and windows were shut during the hot summer months while others visited each other.