Why was Jem bothered by the cement they found in the knot-hole of the oak?

Jem already suspected the gifts might have been from Boo. Mr. Radley's cementing of the knot-hole pretty much confirmed his suspicions, and also alerted him to Boo's situation. Jem is maturing, and finally, seeing that Boo is a person, not merely a story to base games on.