Oliver returns from Rome by train, by himself. He takes in the sights and smells of the Riviera as though arriving for the first time at the start of summer. He finds that his family’s housekeeper has already changed Elio’s room—where Oliver was staying—back to how it was before his arrival; this saddens and infuriates him. Elio’s family members are also sad about Oliver’s absence. Elio tries to return to his usual routine of summer pastimes, but feels pain whenever he remembers Oliver.
At dinner, Oliver calls to confirm that he has arrived safely in New York. Oliver listens on a third line while his parents talk to Oliver. His parents finish their conversation with Oliver and allow Elio to speak with him privately. Elio confesses to Oliver that he doesn’t want to lose him, and they agree to write and call. Oliver plans for a visit around Christmas. He also mentions that he took something from Elio’s room as a memento—when Elio hangs up, he goes and discovers that Oliver took a postcard depicting Monet’s berm.
Elio’s father asks him about Rome, and Elio speaks positively about his experience but withholds information about his affair with Oliver. His father tells him that he knows about their relationship: he tells him that what he had with Oliver was special and rare.
Oliver visits for Christmas and announces that he might be getting married in the spring. Elio is dumbfounded but tries to be happy for Oliver. Oliver still has feelings for Elio and kisses him once, but he decides against doing anything further. That week, Oliver helps the family select the next year’s fellow. Elio reflects on the past fellows and on the first time he saw Oliver’s passport picture attached to his application, wondering if he felt anything back then.
Elio hears of Oliver’s marriage the following summer. Nine years after receiving Oliver’s last letter, he receives a phone call from Oliver in which he hears Oliver’s two sons in the background. Elio reflects on the nature of time and sentimentality.
Four years later, Elio visits ones of Oliver’s lectures in a New England college and waits until all students have left to speak to him. Oliver is overjoyed to see Elio again, and they continue to talk in his office, where Elio notices the framed postcard of the Monet berm. Oliver invites Elio to dinner with his family, but Elio declines, fearing that he might become emotional if he were to do that. Instead, Elio invites him to his hotel—not for intimacy, but to drink at the hotel bar. Oliver accepts; while drinking martinis it becomes clear that their love for one another is still alive. They recall their favorite memories together during their trip to Rome. Elio offers to walk Oliver to his car and Oliver invites him again to for dinner, but Elio refuses once more.
Elio narrates the final scene in a time after his father’s death, twenty years after the events of his summer with Oliver. Oliver visits Elio’s villa on the Italian Riviera once more for an overnight. Elio shows Oliver around the house and points out what he considers to be his father’s "ghost spot"—where his old breakfast table used to be, where he feels his memory most strongly. Oliver asks if he has a ghost spot; Elio thinks to himself that the entire villa will always be his ghost spot.
Oliver says that, like Elio, he remembers everything. Elio concludes by wanting to say that if Oliver is indeed like him and does remember everything, then when he boards his cab the following morning, when there is nothing more to say—even if just in jest—he should hold his gaze and call him by his name, as he did on their first night together.
Themes of memory and timelessness come to a head in the conclusion of the novel. When Elio returns from Rome, he takes in the sights of the Italian Riviera as though arriving for the first time at the beginning of summer. He returns hom—which his servant Mafalda has restored to its original state cleaning all trace of Oliver—and spends his day doing his usual pastimes, but Oliver's absence is nearly tangible. The title of the chapter is "Ghost Spots," which refer to places so evocative of a person's memory that they nearly feel haunted by their spectral presence. At the end of the novel, after Elio's father passes away, his father's ghost spot is his old breakfast table. Oliver's ghost spot, however, is the entire villa and surrounding grounds. Although this is stated explicitly later in the chapter, the concept is felt immediately upon Elio's return to the villa. Oliver's presence was powerful while he was in Italy, but his absence is now felt just as powerfully. Elio can never return to the days before meeting Oliver, as much as he tries to return to his routines and avoid thinking about Oliver.
Both Elio and Oliver hold on to keepsakes to keep each other in their memory. Elio holds on to Oliver's blue shirt, which he nicknames "Billowy." He keeps it in a sealed bag and takes care not to corrupt the smell it carries—the smell of Oliver—which Elio holds on to in order to remember Oliver, a Proustian gesture. Oliver's keepsake is a postcard taken from Elio's room depicting Monet's berm. Although their love affair does not continue past the summer and Oliver goes on to marry a woman and father two sons, both of these objects keep their memory of the love affair alive eternally. They signify the central theme of the novel's concluding chapter: memories are timeless and can be endlessly revisited.
When Elio speaks with his father and his father reveals knowledge and approval of the affair, Elio wonders what judgment his father will render: "He might then add his usual bromide about how rare good friendships were...No man is an island, can't shut yourself away from others, people need people, blah, blah." This harkens back to the first chapter of the novel, in which Elio described how his parents urged him to spend more time out at night with friends and learn "why others are so important." After his experience with Oliver, Elio finally understands this lesson but can't bear to think about it in the moment from the pain of missing Oliver. However, his father offers an insight that Elio hadn't considered: "You're too smart not to know how rare, how special, what you two had was." Oliver ponders for the first time that the nature of his relationship with Oliver was something extraordinary, not even something all humans get to experience. The rest of the chapter, as he details the course of his adult life, shows Elio coming to learn this lesson with every passing year and every consequent relationship.
To sum up his relationship, in response to his father, Elio says, "Oliver was Oliver." This short, simple, obvious tautology summarizes the entire novel. The novel gains its title from the game that the lovers play on their first night together, of calling each other by each other's names, symbolic of Heraclitus's unity of opposites: two lovers come together as one, for the desire to have someone is the same as the desire to be that person. The last words of the book reference this again, taking place twenty years after the events of their summer together: "if you really are like me, then before you leave tomorrow...look me in the face, hold my gaze, and call me by your name."