Oskar Schell is a nine-year-old boy whose father, Thomas Schell, died in the terrorist attacks on the World Trade Center on September 11, 2001. The novel begins after the tragedy, with Oskar narrating. Since his father's death, Oskar struggles with insomnia, panic attacks, and depression. He often describes the feeling of depression as wearing heavy boots, and deals with this by giving himself bruises. His relationship with his mother has also become strained, particularly as she has started dating a man named Ron, whom he resents for replacing his father.
One day, in his father's closet, Oskar finds a key in a small envelope inside a vase that he accidentally broke; in the keyshop he finds the name Black and thinks this has something to do with the key. Curious, Oskar sets out on a mission to contact every person in New York City with the last name of Black in the hope of finding the lock that belongs to the key his father left behind, creating a binder with mementos of his journey.
One of the first people Oskar meets is a 48-year-old woman named Abby Black. Oskar and Abby instantly become friends, but she has no information on the key. Oskar continues to search the city. Toward the end of his journey Oskar meets an old man he calls "the renter" because until the point of meeting, Oskar had only heard of the old man's existence from his grandmother who referred to him as the new tenant in her apartment. The reader learns towards the end of the book that "the renter" is actually Oskar's grandfather, who abandoned his grandmother while she was pregnant with Thomas, though Oskar doesn't realise the connection.
The book spans many months of Oskar's journey, some of which he is accompanied by his eccentric elderly neighbour, Mr. A. Black, and they develop a close friendship. After meeting with a woman named Ruth in the Empire State Building, who has a history of her own with the building, Mr. Black ends his travels with Oskar, who struggles with his emotions inside. He tries to visit Mr. Black again later but finds out he has moved house, presumably to be with Ruth, and is selling his apartment, leaving behind a card for Oskar, reading "Oskar Schell: Son". Eight months after Oskar initially met Abby, he finds a message from her on the answering machine. Oskar had not touched that phone since his father died because his father's last words had been on an identical answering machine which Oskar had kept hidden from his mother. Oskar finds out that Abby called him directly after his visit, saying she wasn't completely honest with him and might be able to help. Oskar returns to Abby's apartment after listening to this message, and Abby directs him to her ex-husband, William Black.
When Oskar talks to William Black, he learns that the key once belonged to William's father. In his will, William's father left William a key to a safe-deposit box, but William had already sold the vase at the estate sale to Thomas Schell. Then, Oskar tells William something that he "never told anyone" – the story of the last answering machine message Oskar received from his father, during the attack of 9/11. Oskar then gives William Black the key. Disappointed that the key does not belong to him, Oskar goes home angry and sad, not interested in the contents of the box. Oskar also discovers that his mother knew about his activities the entire time and was contacting everyone with the name Black in New York City. After the first few visits she called every Black that he would meet and informed them that Oskar was going to visit and why. In response, the people Oskar met knew ahead of time why he was coming and usually treated him in a friendly manner.
Resolving to try to move on from his father's death, Oskar bonds with Ron after finding out that he met his mother at a support group after having lost his wife and daughter in a car accident. On the second anniversary of his father's death, Oskar meets with "the renter" and they go to dig up his father's grave. Contemplating on what to put into the empty coffin, "the renter" decides to bury various letters that he had written to his unborn son. Shortly after returning home, Oskar reconciles with his mother and vows to become better and allow for her to find happiness again, and she tells him how Oskar's father lied to her in his last call, telling her that he was coming home, to assure her not to worry over his death. Before going to bed, Oskar takes out his binder and proceeds to rearrange the pages in reverse in an attempt to relive the last few hours with his father and achieve closure.
The novel has a parallel narrative that eventually converges with the main story. This narrative is portrayed through a series of letters written by Oskar's grandfather to Oskar's father Thomas, and by Oskar's grandmother to Oskar himself. The letters written by Oskar's grandfather explain his past in World War II, his first love, and his marriage to Oskar's grandmother. The letters written by Oskar's grandmother explain her past in meeting Oskar's grandfather, the trouble in their relationship, and how important Oskar is to her. Upon learning of his son's death, Oskar's grandfather promptly returns to New York and tracks down Oskar and his grandmother. His grandmother decides to let him live with her in her apartment temporarily, which results in them becoming intimate, and he watches over Oskar from afar before meeting him. Shortly after burying the letters with Oskar, his grandfather returns to the airport where Oskar's grandmother follows him. After discussing the war, losing their loved ones and their marriage, they decide to stay in the airport for a while.
The final pages are a flip-book style animation of a photograph of a man falling from the World Trade Center. The animation makes the man appear to fall upwards.