Berry Hamilton, an emancipated black man, works as a butler for a wealthy white man Maurice Oakley. Berry lives in a small cottage a short distance away from the Oakley's place of residence. Berry lives with his wife, Fannie, and two children, Joe and Kitty. During a farewell dinner for Maurice's younger brother, Francis Oakley, it becomes known that a large sum of money has disappeared from Oakley residence due to Francis apparently being careless and leaving the key in the safe. Maurice soon convinces himself that Berry must have stolen the money. A court finds Berry guilty of the theft and sentences him to ten years of hard labor.
Maurice and his wife expel Fannie, Joe, and Kitty from the cottage. Unable to find work, Fannie and her children decide to move to New York. Once in New York, Joe begins work and starts regularly visiting the Banner Club. He begins dating an entertainer from the club named Hattie Sterling. To Fannie's disapproval, Hattie helps Kitty to find employment as a singer and actress. Joe's situation quickly declines and he becomes an alcoholic. Hattie breaks the relationship. Completely degraded, Joe strangles Hattie. Later, he confesses to the murder and finds himself in prison. With her husband and son in prison, Fannie is distraught. Kitty convinces Fannie to marry a man named Mr. Gibson.
Francis Oakley, who left for Paris to become an artist, sends a message to Maurice Oakley. When Maurice receives the letter, he postulates that it could be a message informing him of the artistic successes of Francis. To his dismay, it describes how Francis stole the money and he wishes for Berry Hamilton to be released from prison. Maurice decides that he will not announce Berry's innocence in hopes of preserving the honor of his brother and himself.
Mr. Skaggs, an acquaintance of Joe at the Banner Club, overhears the story of Berry Hamilton's conviction for theft. As a writer for New York's Universe, Mr. Skaggs postulates that if he can prove Berry's innocence, he will have a popular article for the publisher. He travels to the hometown of the Hamilton's to converse with Maurice Oakley. He first meets with a man named Colonel Saunders who tells him that he believes Berry is innocent, the money was simply lost, and to protect the secret, Maurice Oakley carries the money in his "secret" pocket at all times. To gain entry into the Oakley residence, Skaggs lies about having a letter from Francis. Mr. Skaggs forcibly removes Francis's letter from Maurice's secret pocket.
With Francis's letter, Mr. Skaggs is able to have Berry pardoned after five years in prison. Mr. Skaggs brings Berry to New York. Soon, Berry finds out about his son, daughter, and wife's new husband. Hopeless, Berry plans to murder his wife's suitor. To Berry's fortune, he finds that Mr. Gibson has been killed in a fight at a racetrack. Broken down by the hardships of the city, Fannie and Berry decide to move back to the cottage near the Oakley residence when the apologetic Mrs. Oakley begs them to return.