A wealthy young English gentleman, Louis Trevelyan, visits the fictional Mandarin Islands, a distant British possession, and becomes smitten with Emily Rowley, the eldest daughter of the governor, Sir Marmaduke Rowley. The Rowleys accompany Trevelyan to London, where he marries Emily. When the rest of the family goes home, Emily's sister Nora remains behind, under Trevelyan's protection.
The marriage is initially a happy one and the couple have a baby boy. Then a seemingly minor matter undermines their marriage. Colonel Osborne, an old friend of Sir Marmaduke's, visits Emily much too frequently for her husband's taste. Though nothing improper occurs, Trevelyan orders his wife to avoid the man in future. Emily resents his lack of trust and makes no attempt to hide it. Their relationship deteriorates to the point that they separate.
Meanwhile, Nora attracts two admirers, the wealthy Charles Glascock, the eldest son and heir of Lord Peterborough, and Hugh Stanbury, a close friend of Trevelyan's from their days at Oxford University. Stanbury ekes out a precarious living writing newspaper articles. Glascock proposes to Nora, but despite the fact that Stanbury has given no indication of his feelings for her, she rejects the future nobleman, not without a great deal of struggle and much to the dismay of her friends.
Another subplot involves Jemima Stanbury, the capricious, formidable spinster aunt of Hugh. In her youth, she had been engaged to the eldest son of a leading banker. They had had a falling out and parted company, but upon his demise, he had left everything to her, making her very wealthy. Aware of the poverty of Hugh's branch of the family, she had generously paid for his education and helped him get a start in life. However, when he chose to work for what she considered to be a radical publication, the staunch Tory withdrew her support. She then offers to accept one of Hugh's sisters as a companion. After some debate, the timid and unassertive Dorothy Stanbury is sent to live with her.
Trevelyan arranges to have Emily and Nora live with Hugh's mother and her other daughter, Priscilla. However, Emily obstinately receives a visit from Colonel Osborne, against all advice to the contrary. Trevelyan finds out and becomes further maddened.
In the meantime, Aunt Stanbury tries to promote a marriage between her niece Dorothy and a favoured clergyman, Mr Gibson. This causes much resentment with Arabella and Camilla French, two sisters who had considered him a future husband for one of them (though which was still a matter of much debate). However, this plan is derailed.
Aunt Stanbury had always intended to bequeath her wealth back to the Burgess family, rather than to her Stanbury relations. She had chosen as her heir Brooke Burgess, the nephew of her former fiancé. When he visits her for the first time as an adult, everyone is charmed by his warm, lively personality, especially Dorothy. When Gibson finally proposes to her, she cannot avoid unfavourably comparing him to Brooke and declines. Her aunt is at first much put out by Dorothy's obstinacy. Eventually however, she places the blame on the clergyman, which results in a serious breach between them.
The feud with his former patron leaves Gibson so distracted that he finds himself engaged to a domineering Camilla French. After a while, he comes to regret his choice. Finally, finding Camilla's overpowering personality unbearable, he extricates himself by agreeing to marry the milder Arabella instead. Camilla is driven to extravagant threats and is finally sent to stay with her stern uncle in the period leading up to the wedding.
Then Aunt Stanbury becomes very ill, resulting in Dorothy and Brooke spending a good deal of time in each other's company. Brooke takes the opportunity to propose to an unsuspecting Dorothy. She however is reluctant to accept, fearing that her aunt will disinherit Brooke. Instead, the old woman blames her niece. They quarrel and Dorothy returns to her mother.
Aunt Stanbury misses Dorothy greatly and makes it known that she would welcome her back, though she still vehemently opposes her marriage to Brooke. Dorothy does come back, and even tries to break off her engagement, but Brooke will not stand for it. In the end, Aunt Stanbury's love for her niece is stronger than her desires and she gives her blessing to their wedding.
Meanwhile, Trevelyan departs England to escape the shame he feels. During his aimless wanderings, he meets Mr Glascock, who is on his way to Italy to visit his father. They encounter two attractive young American ladies, Caroline and Olivia Spalding. Glascock's father is in such poor health that the son is obliged to remain in the country to await his probable demise. While waiting, he courts and wins Caroline's hand in marriage, despite her misgivings about her reception in English society.
Trevelyan receives word that Colonel Osborne has dared to visit Emily once again. While Osborne had not been permitted to see Emily, Trevelyan does not believe it and has the boy taken away from his mother by deception; he takes his son back to Italy, where he descends further into madness. Eventually, he is tracked down by his wife and friends. Emily persuades him first to give her their son, then to return with her to England; he dies, however, shortly after their return. In his dying moments, Emily begs Louis to kiss her hand to signify that he does not believe she did anything wrong. Whether or not he does is unclear, but Emily believes "the verdict of the dying man had been given in her favour." A few months later, Hugh Stanbury and Nora marry.
