Why doesn't the government track down the men in the camps?

From the text:

They survive, I guess. There aren't many of them, and I guess the Government's never considered them a great enough danger to go in and track them down.