In June 1642, in the Puritan town of Boston, a crowd gathers to witness the punishment of Hester Prynne, a young woman found guilty of adultery. She is required to wear a scarlet "A" ("A" standing for adulterer) on her dress to shame her. She must stand on the scaffold for three hours, to be exposed to public humiliation. As Hester approaches the scaffold, many of the women in the crowd are angered by her beauty and quiet dignity. When demanded and cajoled to name the father of her child, Hester refuses.
As Hester looks out over the crowd, she notices a small, misshapen man and recognizes him as her long-lost husband, who has been presumed lost at sea. When the husband sees Hester's shame, he asks a man in the crowd about her and is told the story of his wife's adultery. He angrily exclaims that the child's father, the partner in the adulterous act, should also be punished and vows to find the man. He chooses a new name – Roger Chillingworth – to aid him in his plan.
Reverend John Wilson and the minister of Hester's church, Arthur Dimmesdale, question the woman, but she refuses to name her lover. After she returns to her prison cell, the jailer brings in Roger Chillingworth, a physician, to calm Hester and her child with his roots and herbs. He and Hester have an open conversation regarding their marriage and the fact that they were both in the wrong. Her lover, however, is another matter and he demands to know who it is; Hester refuses to divulge such information. He accepts this, stating that he will find out anyway, and forces her to hide that he is her husband. If she ever reveals him, he warns her, he will destroy the child's father. Hester agrees to Chillingworth's terms although she suspects she will regret it.
Following her release from prison, Hester settles in a cottage at the edge of town and earns a meager living with her needlework. She lives a quiet, somber life with her daughter, Pearl. She is troubled by her daughter's unusual fascination by Hester's scarlet "A". As she grows older, Pearl becomes capricious and unruly. Her conduct starts rumors, and, not surprisingly, the church members suggest Pearl be taken away from Hester.
Hester, hearing rumors that she may lose Pearl, goes to speak to Governor Bellingham. With him are Reverends Wilson and Dimmesdale. Hester appeals to Reverend Dimmesdale in desperation, and the minister persuades the governor to let Pearl remain in Hester's care.
Because Dimmesdale's health has begun to fail, the townspeople are happy to have Chillingworth, a newly arrived physician, take up lodgings with their beloved minister. Being in such close contact with Dimmesdale, Chillingworth begins to suspect that the minister's illness is the result of some unconfessed guilt. He applies psychological pressure to the minister because he suspects Dimmesdale to be Pearl's father. One evening, pulling the sleeping Dimmesdale's vestment aside, Chillingworth sees a symbol that represents his shame on the minister's pale chest.
Tormented by his guilty conscience, Dimmesdale goes to the square where Hester was punished years earlier. Climbing the scaffold, he admits his guilt to them but cannot find the courage to do so publicly. Hester, shocked by Dimmesdale's deterioration, decides to obtain a release from her vow of silence to her husband.
Several days later, Hester meets Dimmesdale in the forest and tells him of her husband and his desire for revenge. She convinces Dimmesdale to leave Boston in secret on a ship to Europe where they can start life anew. Renewed by this plan, the minister seems to gain new energy. On Election Day, Dimmesdale gives what is declared to be one of his most inspired sermons. But as the procession leaves the church, Dimmesdale climbs upon the scaffold and confesses his sin, dying in Hester's arms. Later, most witnesses swear that they saw a stigma in the form of a scarlet "A" upon his chest, although some deny this statement. Chillingworth, losing his will for revenge, dies shortly thereafter and leaves Pearl a substantial inheritance.
Several years later, Hester returns to her cottage and resumes wearing the scarlet letter. When she dies, she is buried near the grave of Dimmesdale, and they share a simple slate tombstone engraved with an escutcheon described as: "On a field, sable. The letter A, gules".