According to Faber, why is Faber a ''coward''

From the text:

"Mr. Montag, you are looking at a coward. I saw the way things were going, along time back. I said nothing. I'm one of the innocents who could have spoken up and out when no one would listen to the `guilty,' but I did not speak and thus became guilty myself. And when finally they set the structure to burn the books, using the, firemen, I grunted a few times and

subsided, for there were no others grunting or yelling with me, by then. Now, it's too late."