Herzog is set in 1964 in the United States, and is about the midlife crisis of a Jewish man named Moses E. Herzog. At the age of forty-seven, he is just emerging from his second divorce, this one particularly acrimonious. He has two children, one by each wife, who are growing up without him. His career as a writer and an academic has floundered. He is in a relationship with a vibrant woman, Ramona, but finds himself running away from commitment.
Herzog's second marriage, to the demanding, manipulative Madeleine, has recently ended in a humiliating fashion. While still actively married, Madeleine convinced Moses to move her and their daughter Junie to Chicago, and to arrange for their best friends, Valentine and Phoebe Gersbach, to move as well, securing a solid job for Valentine. However, the plans were all a ruse, as Madeleine and Valentine were carrying on an affair behind Moses's back, and shortly after arriving in Chicago, Madeleine throws Herzog out, securing a restraining order (of sorts) against him, and attempting to have him committed to an asylum.
Herzog spends much of his time mentally writing letters he never sends. These letters are aimed at friends, family members, and famous figures. The recipients may be dead, and Herzog has often never met them. The one common thread is that Herzog is always expressing disappointment, either his own in the failings of others or their words, or apologizing for the way he has disappointed others.
The novel opens with Herzog in his house in Ludeyville, a town in the Berkshires in western Massachusetts. He is contemplating returning to New York to see Ramona, but instead flees to Martha's Vineyard to visit some friends. He arrives at their house, but writes a note – this one an actual note – saying that he has to leave:
- "Not able to stand kindness at this time. Feeling, heart, everything in strange condition. Unfinished business."
He heads to New York to start trying to finish that business, which includes regaining custody of his daughter Junie. After spending a night with Ramona, he heads to the courthouse to discuss his plans with his lawyer. He ends up witnessing a series of tragicomic court hearings, including one where a woman is charged with beating her three-year-old to death by flinging him against a wall. Moses, already distraught after receiving a letter from Junie's babysitter about an incident where Valentine locked Junie in the car while he and Madeleine argued inside the house, heads to Chicago. He goes to his stepmother's house and picks up an antique pistol with two bullets in it, forming a vague plan of killing Madeleine and Valentine and running off with Junie.
The plan goes awry when he sees Valentine giving Junie a bath and realizes that Junie is in no danger. The next day, after taking his daughter to the aquarium, Herzog crashes his car and ends up being charged with possession of a loaded weapon. His brother, the rational Will, picks him up and tries to get him back on his feet. Herzog heads to Ludeyville, where his brother meets him and tries to convince him to check himself into an institution. But Herzog, who had previously considered doing just that, is now coming to terms with his life. Ramona comes up to join him for dinner – much to Will's surprise – and Herzog begins making plans to fix up the house, which, like his life, needs repair but is still structurally sound. Herzog closes by saying that he doesn't need to write any more letters.
Through the flashbacks that litter the novel, other critical details of Herzog's life come to light, including his marriage to the stable Daisy and the existence of their son, Marco; the life of Herzog's father, a failure at every job he tried; and Herzog's sexual molestation by a stranger on a street in Montreal.