Jeannette Walls is the second oldest of four children. Her father, Rex, is an alcoholic, and her mother, Rose Mary, a painter and teacher. Until Jeannette is six, the family moves around Arizona and California every few months when Rex's and Rose Mary's debts grow too numerous. When Jeannette is seven, the family moves to Battle Mountain, Nevada where they enjoy stability for the first time as Rex works for a mining company and the family lives in a converted railway station. Eventually Rex loses his job and the children grow hungry. Rose Mary, who has a teaching certificate, gets a job teaching at the local school, but Rex quickly siphons away her paycheck. Even so, the family is happy there until a young boy develops a fixation on Jeannette and attacks her with a BB gun when the children are home alone. Jeannette's older sister Lori retrieves their father's pistol to scare him away but the police are called and when Rex and Rose Mary learn that the children might be taken away from them, they decide to flee to Phoenix, Arizona. Jeannette initially believes they are moving to live with her maternal grandmother, but on the way she is informed Grandma Smith has died and that they are going to live on the property Rose Mary has inherited from her mother.
At first, life is happy for the children. Their mother's house is big, and Grandma Smith also left her a significant amount of money. However, the money quickly disappears and the house falls into a state of disrepair. Rex asks Jeannette what she would like for her 10th birthday, and Jeannette says she would like him to stop drinking. He ties himself to a bed for a week to overcome his addiction to alcohol, then decides to take the family on a trip to the desert. When their car breaks down in the desert, a woman who picks them up and takes them to the city refers to them as "poor", causing Rex to relapse. Rose Mary decides that since they have no money it is time to move again, and she takes the family to their paternal grandparents's in Welch, West Virginia.
In Welch the children meet their paternal grandparents and uncle for the first time. They are enrolled in school; however, since Rose Mary abandoned their records, and the children have accents unfamiliar to the locals, they are placed in a class for challenged children. Jeannette is repeatedly beat up by local girls, but when she helps the neighbor of the lead bully, she is no longer targeted. Rex and Rose Mary decide to return to Phoenix to retrieve some valuable items they abandoned. While they are gone, Jeannette walks in on her grandmother molesting Brian. Lori gets into a physical altercation with their grandmother, and she and Brian realize their father was probably molested as well. Jeannette and Lori became upset. When Rex returns, he admonishes his children rather than defending them, but the family is told to leave. They relocate to a small rotting house with no indoor plumbing, on land that Rex acquires with the intention of building his dream house, a glass castle.
Rex assures his children that their situation is temporary, but they live at the house for years as it falls further into disarray and Rex refuses to repair it. Their only money comes from the odd jobs Rex finds, and the infrequent checks Rose Mary receives from an oil company leasing a piece of property she owns. The children resort to dumpster diving to survive. Jeannette begs her mother to leave her father so they can go on welfare, but her mother refuses. Eventually Rose Mary takes a teaching job after a man from child protective services pays them a visit. The children believe their lives will change after their mother has work, but their money continues to evaporate and their mother suffers nervous breakdowns from the stresses of teaching.
The summer Jeanette is thirteen, her mother leaves to take teaching classes and her sister is away on scholarship. Jeanette gives her father some of the money her mother has left her to run the household. She ends up unwittingly working with her father in a pool hustling scam where she is groped and nearly raped by a much older man, then refuses to participate in any more of her father's schemes. In an effort to find money, she lands her first real job, working at a jewelry store.
When Rose Mary returns from her teaching seminar, she decides to quit teaching to refocus on her art. Disgusted, Lori and Jeannette hatch a plan for Lori to move to New York City with Jeannette following shortly thereafter. Lori, Jeannette and Brian work for the better part of a year to accumulate money for the move. Shortly before Lori is set to move, Jeannette discovers Rex has stolen their money. Lori is disheartened, but Jeannette gets an offer to babysit for the summer. She asks the couple to hire Lori instead, and to buy her a ticket to New York in payment.
Jeannette begins making plans to go to college in New York City, and realizes she can leave a year early and complete 12th grade there. Rose Mary is indifferent to her leaving, but Rex seems heartbroken and accompanies her to the bus station. After graduating from Barnard College, Jeannette gets an internship at a newspaper. She encourages Brian to join her and Lori in New York, and he agrees. When her youngest sister Maureen is twelve, Lori asks her to move in with them as the house in Welch is on the verge of being condemned; Maureen readily agrees. A short while later, Jeannette gets a call from Rose Mary who tells her that she and Rex have moved to the city to be with their children. Though Lori and Brian try to help their parents, they must eventually ban them from their apartments. The parents become homeless and end up living in abandoned buildings. When Maureen enters her twenties, she moves back in with them. A fight eventually breaks out between Maureen and Rose Mary, and Maureen tries to stab Rose Mary. She is arrested and forced to spend a year in a mental institution. When she is released, she decides to move to California.
A few years later, Rex calls Jeannette and tells her that he is dying. He dies a few weeks later. Years later, the family gathers on Thanksgiving where they toast Rex.