Robert Moore is a mill owner noted for apparent ruthlessness towards his employees. He has laid off many of them, and is apparently indifferent to their consequent impoverishment. In fact he had no choice, since the mill is deeply in debt. He is determined to restore his family's honour and fortune.
As the novel opens Robert awaits delivery of new labour-saving machinery for the mill, which will enable him to lay off additional employees. Together with some friends he watches all night, but the machinery is destroyed by "frame-breakers" on the way to the mill. Robert's business difficulties continue, due in part to continuing labour unrest, but even more to the Napoleonic Wars and the accompanying Orders in Council, which forbid British merchants from trading in American markets.
Robert is very close to his cousin Caroline Helstone, who comes to his house to be taught French by his sister, Hortense. Caroline worships Robert. Caroline's father is dead and her mother has abandoned her, leaving her to be brought up by her uncle, Rev. Helstone. To keep himself from falling in love with her Robert keeps his distance, since he cannot afford to marry for pleasure or for love.
Caroline realises that Robert is growing increasingly distant and withdraws into herself. Her uncle does not sympathise with her "fancies". She has no money of her own, so she cannot leave, which is what she longs to do. She suggests that she might take up the role of governess, but her uncle dismisses the idea and assures her that she need not work for a living.
Caroline recovers somewhat when she meets Shirley, an independent heiress whose parents are dead and who lives with Mrs Pryor, her former governess. Shirley is lively, cheerful, full of ideas about how to use her money and how to help people, and very interested in business. Caroline and Shirley soon become close friends. Caroline becomes convinced that Shirley and Robert will marry. Shirley likes Robert, is very interested in his work, and is concerned about him and the threats he receives from laid-off millworkers. Both good and bad former employees are depicted. Some passages show the real suffering of those who were honest workers and can no longer find good employment; other passages show how some people use losing their jobs as an excuse to get drunk, fight with their previous employers, and incite other people to violence. Shirley uses her money to help the poorest, but she is also motivated by the desire to prevent any attack on Robert.
One night Rev. Helstone asks Shirley to stay with Caroline while he is away. Caroline and Shirley realise that an attack on the mill is imminent. They hear the dog barking and realise that a group of rioters has come to a halt outside the rectory. They overhear the rioters talking about entering the house, but are relieved when they decide to go on. The women go to the mill together to warn Robert, but they are too late. They witness the ensuing battle from their hiding place.
The whole neighbourhood becomes convinced that Robert and Shirley will marry. The anticipation of this event causes Caroline to fall ill. Mrs Pryor comes to look after her and learns the cause of Caroline's sorrow. She continues her vigil even as Caroline worsens daily. Mrs Pryor then reveals to Caroline that she is Caroline's mother. She had abandoned her because Caroline looked exactly like her father, the husband who tortured Mrs Pryor and made her life miserable. She had little money, so when her brother-in-law offered to bring up the child, she accepted the offer, took up the name of Pryor and went off to become a governess. Caroline now has a reason to live, since she knows that she can go and live with her mother, and begins to recover.
Shirley's uncle and aunt come to visit her. They bring with them their daughters, their son, and their son's tutor, Louis Moore. He is Robert's younger brother and taught Shirley when she was younger. Caroline is puzzled by Shirley's haughty and formal behaviour towards Louis. Two men fall in love with Shirley and woo her, but she rejects both of them because she does not love them. The relationship between Shirley and Louis, meanwhile, remains ambivalent. There are days when Louis can ask Shirley to come to the schoolroom and recite the French pieces she learned from him when she was younger. On other days Shirley ignores Louis. However, when Shirley is upset the only person she can confide in is Louis. After a supposedly mad dog bites Shirley and makes her think that she is to die early no one except Louis can make her reveal her fears.
Robert returns one dark night, first stopping at the market and then returning to his home with a friend. The friend asks him why he left when it seemed so certain that Shirley loved him and would have married him. Robert replies that he had assumed the same, and that he had proposed to Shirley before he left. Shirley had at first laughed, thinking that he was not serious, and then cried when she discovered that he was. She had told him that she knew that he did not love her, and that he asked for her hand, not for her sake, but for her money. Robert had walked away filled with a sense of humiliation, even as he knew that she was right. This self-disgust had driven Robert away to London, where he realised that restoring the family name was not as important as maintaining his self-respect. He had returned home determined to close the mill if he had to, and go away to Canada to make his fortune. Just as Robert finishes his narration his friend hears a gunshot and Robert falls from his horse.
The friend takes Robert to his own home and looks after him. After a turn for the worse Robert slowly gets better. A visit from Caroline revives him, but she has to come secretly, hiding from her uncle and his friend and his family. Robert soon moves back to his own home and persuades his sister that the very thing their house needs to cheer it up is a visit from Caroline. Robert asks for Caroline's forgiveness.
Louis proposes to Shirley, despite the difference in their relative situations, and Shirley agrees to marry him. At first Caroline is to be Shirley's bridesmaid, but Robert proposes to her and she accepts. The novel ends with Caroline marrying Robert and Shirley marrying Louis.