Atticus abhors violence. Guns are a symbol of violence so Atticus tries to steer his kids in a different direction. Atticus wanted to show his children that real courage has nothing to do with guns and violence.
"I wanted you to see what real courage is, instead of getting the idea that courage is a man with a gun in his hand. It's when you know you're licked before you begin but you begin anyway and you see it through no matter what. You rarely win, but sometimes you do. Mrs Dubose won, all ninety-eight pounds of her. According to her views, she died beholden to nothing and nobody. She was the bravest person I ever knew."