As Dahl's novel begins, the reader is introduced to James, a young boy who is orphaned when his parents are eaten by a rhinoceros. James is consequently sent to live with his two aunts, Aunt Sponge and Aunt Spiker. These relatives are very cruel to him, and he is incredibly lonely, since he has no friends yet longs to play with children his own age. But on one particularly hard day, James's luck changes: an Old Man appears in the backyard garden and offers James a packet of magical green objects. If James follows a set of specific instructions (says the Old Man), something spectacular will happen. James is very excited, but as he runs back to his house to execute the instructions, he trips and the magical green objects burrow into the ground. James is incredibly upset, but as he begins to resume his chores, he hears his aunts shouting.
James soon discovers the source of the commotion: a peach has begun to grow on a top branch of a previously barren peach tree. As the three of them watch, the peach becomes larger and larger, until it is bigger than the aunts' entire house. Seeking to capitalize on this strange event, Aunt Sponge and Aunt Spiker set up a fence and begin charging admission to see the peach. They forbid James from interfering, fearing that he will ruin their profit-making scheme. The night after the first day of visitors, James sneaks out of the house to visit the peach. He sees a hole at the bottom of the peach, and he realizes that this hole is the opening to a tunnel. He begins to crawl through, and he eventually enters the hollow peach pit at the center of the fruit.
When James enters the pit's inner chamber, he meets an odd assortment of creatures, who initially intimidate him: Miss Spider, Centipede, Earthworm, Old-Green-Grasshopper, and others. The next day James and his new companions begin their journey away from Aunt Sponge and Aunt Spiker. Centipede cuts the peach away from its tree and the peach begins to roll, flattening everything in its way - including Aunt Sponge and Aunt Spiker, who are killed. The peach then rolls off of a cliff several miles away and lands in the Atlantic Ocean.
From this point forward, James and his friends face a series of obstacles. They must escape attacking sharks, evade the Cloud-Men and their anger, and settle internal disputes. James asserts himself as the leader and frequently saves the day. When sharks attack, he has his companions fasten ropes to nearby seagulls, then to the stem of the peach, the peach rises out of the water and begins flying through the air. After one day of flying, James and his friends realize that they have flown across the entire Atlantic - they can now see New York City below them.
They begin to cut the seagull strings one-by-one, when suddenly a passenger plane flies above them and cuts all of the lines at once. Their gradual descent into the city is ruined and they begin to sink rapidly. Everyone holds on for dear life, thinking death is imminent, but they land safely on the pinnacle of the Empire State Building. After explaining their situation to the New York City policemen and firemen, James and his friends are brought to street level and are welcomed lavishly. The city throws a parade in their honor, and by the end of the parade the entire peach has been eaten by local children, who want to taste the giant fruit.
After the parade, James and his friends live happily ever after. The remaining peach pit is set up as a monument in Central Park, and James lives inside of it. So many children visit him, hoping to hear his story, that he decides to write a book about the journey. The book that he wrote is the book that the reader has just read.