The story begins with Neddy Merrill lounging at a friend's pool on a warm midsummer day. On a whim, Neddy decides to return home by swimming through all the pools in the neighborhood, which he names "the Lucinda River" to honor his wife. He begins the journey enthusiastic and full of youthful energy and, in the early stops on his journey, his friends enthusiastically greet him with drinks; it is readily apparent that he is well-regarded, and has an upper or upper-middle-class social standing.
As his journey progresses, the story's tone gradually becomes darker and more surreal. Despite the ever-present afternoon light, it becomes unclear how much time has passed: at the beginning of the story it is clearly midsummer, but eventually all natural signs point to the season being autumn. Some old acquaintances Neddy encounters mention his financial problems, although he does not remember having such misfortunes. He is patently unwelcome at several houses belonging to owners of a lower social class. His earlier, youthful energy gradually declines, and it becomes increasingly painful and difficult for him to swim on. Finally, he staggers back home only to find his house decrepit, empty, and abandoned.