In Puritan Boston, Massachusetts, a crowd gathers to witness the punishment of Hester Prynne, a young woman who has given birth to a baby of unknown parentage. She is required to wear a scarlet "A" on her dress when she is in front of the townspeople to shame her. The letter "A" stands for adulteress, although this is never said explicitly in the novel. Her sentence required her to stand on the scaffold for three hours, exposed to public humiliation, and to wear the scarlet "A" for the rest of her life. As Hester approaches the scaffold, many of the women in the crowd are angered by her beauty and quiet dignity. When demanded and cajoled to name the father of her child, Hester refuses.
As Hester looks out over the crowd, she notices a small, misshapen man and recognizes him as her long-lost husband, who has been presumed lost at sea. When the husband sees Hester's shame, he asks a man in the crowd about her and is told the story of his wife's adultery. He angrily exclaims that the child's father, the partner in the adulterous act, should also be punished and vows to find the man. He chooses a new name, Roger Chillingworth, to aid him in his plan.
The Reverend John Wilson and the minister of Hester's church, Arthur Dimmesdale, question her, but she refuses to name her lover. After she returns to her prison cell, the jailer brings in Roger Chillingworth, a physician, to calm Hester and her child with his roots and herbs. He and Hester have an open conversation regarding their marriage and the fact that they were both in the wrong. Her lover, however, is another matter and he demands to know who it is; Hester refuses to divulge such information. He accepts this, stating that he will find out anyway, and forces her to conceal that he is her husband. If she ever reveals him, he warns her, he will destroy the child's father. Hester agrees to Chillingworth's terms although she suspects she will regret it.
Following her release from prison, Hester settles in a cottage at the edge of town and earns a meager living with her needlework, which is of extraordinary quality. She lives a quiet, somber life with her daughter, Pearl, and performs acts of charity for the poor. She is troubled by her daughter's unusual fascination with the scarlet "A". The shunning of Hester also extends to Pearl, who has no playmates or friends except her mother. As she grows older, Pearl becomes capricious and unruly. Her conduct starts rumors, and, not surprisingly, the church members suggest Pearl be taken away from Hester.
Hester, hearing rumors that she may lose Pearl, goes to speak to Governor Bellingham. With him are ministers Wilson and Dimmesdale. Hester appeals to Dimmesdale in desperation, and the minister persuades the governor to let Pearl remain in Hester's care.
Because Dimmesdale's health has begun to fail, the townspeople are happy to have Chillingworth, the newly arrived physician, take up lodgings with their beloved minister. Being in such close contact with Dimmesdale, Chillingworth begins to suspect that the minister's illness is the result of some unconfessed guilt. He applies psychological pressure to the minister because he suspects Dimmesdale is Pearl's father. One evening, pulling the sleeping Dimmesdale's vestment aside, Chillingworth sees a symbol that represents his shame on the minister's pale chest.
Tormented by his guilty conscience, Dimmesdale goes to the square where Hester was punished years earlier. Climbing the scaffold in the dead of night, he admits his guilt but cannot find the courage to do so publicly in the light of day. Hester, shocked by Dimmesdale's deterioration, decides to obtain a release from her vow of silence to her husband.
Several days later, Hester meets Dimmesdale in the forest and tells him of her husband and his desire for revenge. She convinces Dimmesdale to leave Boston in secret on a ship to Europe where they can start life anew. Inspired by this plan, the minister seems to gain new energy. On Election Day, Dimmesdale gives one of his most inspired sermons. But as the procession leaves the church, Dimmesdale climbs upon the scaffold and confesses his sin, dying in Hester's arms. Later, most witnesses swear that they saw a stigma in the form of a scarlet "A" upon his chest, although some deny this statement. Chillingworth, losing his will for revenge, dies shortly thereafter and leaves Pearl a substantial inheritance.
After several years, Hester returns to her cottage and resumes wearing the scarlet letter. When she dies, she is buried near the grave of Dimmesdale, and they share a simple slate tombstone engraved with an escutcheon described as: "On a field, sable, the letter A, gules" ("A red letter A written on a black background").