Why do Mr. Pankhurst and the others leave Poker Flat?

The characters are actually forced to leave. After a string of robberies in town, prominent people used Mr. Oakhurst, the Duchess, Mother Shipton, and Uncle Billy as scapegoats to rid the town of "improper persons".