In 1957 suburban Connecticut, Cathy Whitaker seems to be the perfect wife, mother, and homemaker. Her husband, Frank, is an executive at Magnatech, a television advertising company. One evening, Cathy receives a phone call from the local police, who are holding Frank. When she picks him up, he says it is all a misunderstanding, but it turns out that he has been exploring the illicit underground world of gay bars in Hartford. In the midst of all of this, one day Cathy sees an unfamiliar black man walking in her yard. He turns out to be Raymond Deagan, the son of her late gardener, who is taking over his father's accounts.
Frank often stays late at the office. One night when he says he is working, Cathy decides to bring him dinner, and she walks in on him passionately kissing a man. He confesses to having had "problems" as a young man and agrees to sign up for conversion therapy. However, their relationship is strained, Frank's work suffers, and he increasingly turns to alcohol. Cathy runs into Raymond at a local art show and, to the consternation of onlookers, initiates a discussion with him about modern painting. After a party, Frank attempts to make love to Cathy, but he is unable to become aroused and accidentally strikes her when she tries to console him.
The next day, Raymond catches Cathy crying and asks her to run some errands with him. She agrees, and they wind up going to a bar in a black neighborhood, where Cathy is the only white person present. They are seen together on the street by a gossipy acquaintance of Cathy's, and the woman immediately begins to stir up scandal. When Cathy attends her daughter's ballet performance, the mothers of the other girls shun her. Frank also hears about Cathy and Raymond and, in response to his anger, Cathy denies having an other-than-professional relationship with Raymond and says she has fired him to quell the rumors. She then tells Raymond that their friendship cannot continue, as it is not "plausible".
Cathy and Frank go to Miami for New Year's to try to repair their marriage, but, at the hotel, Frank meets a young man and has another homosexual encounter. While they are gone, three white boys taunt and physically assault Raymond's daughter, Sarah, partly because of her father's rumored relationship with Cathy, and the girl is concussed by a rock thrown at her head.
Frank breaks down and tearfully tells Cathy that he has fallen in love with a man and wants a divorce. When Cathy learns what happened to Sarah, she visits Raymond, who says he is moving to Baltimore in two weeks, as the rumors have destroyed his business and led his African-American neighbors to throw rocks through his windows. When Cathy tells Raymond she is going to be single and asks if she can come visit him some time, he stoically, but gently, rejects her, saying he has learned his lesson and needs to do what is right for his daughter.
Cathy shows up at the train station to see Raymond off, and they silently wave to each other as the train leaves the station.
