What is the meaning of the title?

The narrator, who is bi-racial and whose skin color is light enough that he can "pass," has many goals in the novel.  Some can be accomplished while interacting with members of the Black community; however, he has to act as if he is not Black (an ex-colored man) if he is to accomplish his other goals.  He has to behave as an ex-colored man part of the time.