The novel is split into five "books," each covering a different period in the characters' lives. Book One opens in 1912 and introduces 11-year-old Francie Nolan, who lives in the Williamsburg tenement neighborhood of Brooklyn with her 10-year-old brother Cornelius ("Neeley" for short) and their parents, Johnny and Katie. Francie relies on her imagination and her love of reading to provide a temporary escape from the poverty that defines her daily existence. The family subsists on Katie's wages from cleaning apartment buildings, pennies from the children's junk-selling and odd jobs, and Johnny's irregular earnings as a singing waiter. His alcoholism has made it difficult for him to hold a steady job, and he sees himself as a disappointment to his family as a result. Francie admires him because he is handsome, talented, emotional and sentimental, like her. Francie's mother, Katie, has very little time for sentiment, since she is the breadwinner of the family who has forsaken fantasies and dreams for survival.
Book Two jumps back to 1900, with the meeting of Johnny and Katie, the teenage children of immigrants from Ireland and Austria, respectively. Although Johnny panics when Katie becomes pregnant with first Francie and then Neeley, and begins drinking heavily, Katie resolves to give her children a better life than she has known, which her mother, Mary, embodies in the word and idea, "education." Kate also resents baby Francie because she is constantly ill, while Neeley is more robust. Kate makes a promise to herself that her daughter must never learn of her preference for Neeley. During the first seven years of their marriage, the Nolans are forced to move twice within Williamsburg, due to public disgrace brought about first by Johnny's drunkenness and subsequent delirium tremens on his "voting birthday" and then by the children's Aunt Sissy's misguided efforts at babysitting them, both of which set neighbors' tongues to talking. They arrive at the apartment introduced in Book One.
In Book Three, the Nolans settle into their new home and the children (now seven and six) begin to attend the squalid, overcrowded public school next door. Francie enjoys learning even in these dismal surroundings, and with her father's support, she gets herself transferred to a better school in a different neighborhood. Johnny's attempts to improve the children's minds fail, but Katie helps Francie grow as a person and saves her life by shooting a child-rapist/murderer who tries to attack her daughter shortly before she turns 14. When Johnny learns that Katie is pregnant once again, he falls into a depression that leads to his death from alcoholism-induced pneumonia on Christmas Day 1915. Money from the family's life insurance policies is taken by an unscrupulous undertaker. The children's after-school jobs keep the Nolans afloat in 1916 until the new baby, Annie Laurie (named after a favorite song of Johnny's), is born in May and Francie graduates from grade school in June. Graduation allows her to finally come to terms with the reality of her father’s death.
At the start of Book Four, Francie and Neeley take jobs since there is no money to send them to high school. Francie works first in an artificial-flower factory, then, after lying about her age, gets a well paying job ($15 per week) in a press clipping office. Although she wants to use her salary to start high school in the fall, Katie decides to send Neeley instead, reasoning that he will only continue learning if he is forced into it while Francie will find a way to do it on her own. Once the United States enters World War I in 1917, the clipping office rapidly declines and closes, leaving Francie out of a job. After she finds work as a teletype operator, she makes a new plan for her education, choosing to skip high school and take summer college-level courses. She passes with the help of Ben Blake, a friendly and determined high school student, but fails the college's entrance exams. A brief encounter with Lee Rhynor, a soldier about to ship out to France, leads to heartbreak after he pretends to be in love with Francie when he is in fact about to get married. In 1918, Katie accepts a marriage proposal from Michael McShane, a pipe-smoking retired police officer who has long admired Katie, and has meanwhile become a wealthy businessman and politician.
As Book Five begins in the fall of this same year, Francie, now almost 17, quits her teletype job. She is about to start classes at the University of Michigan, having passed the entrance exams with Ben's help, and is considering the possibility of a future relationship with him. The Nolans prepare for Katie's wedding and the move from their Brooklyn apartment to McShane's home, and Francie pays one last visit to some of her favorite childhood places and reflects on all the people who have come and gone in her life. She is struck by how much of Johnny's character lives on in Neeley, who has become a talented jazz/ragtime piano player. Before she leaves the apartment, she notices the Tree of Heaven that has grown and re-sprouted in the building's yard despite all efforts to destroy it, seeing in it a metaphor for her family's ability to overcome adversity and thrive. In the habits of a neighborhood girl, Florrie, she sees a version of her young self, sitting on the fire escape with a book and watching the young ladies of the neighborhood prepare for their dates. Francie says, "Hello, Francie" to Florrie, and then, "Goodbye, Francie" softly, as she closes the window.