What does Charlie say about his dad hitting him and his siblings?

When Charlie first addresses his father slapping him, we immediately take for granted that he's abusive. In this case, it just isn't so, as Charlie later cites that his father "felt terrible for doing it. And he was so sorry. And he would never hit me again. And he hasn't"