The story is told from the perspective of Vandyck "Van" Jennings, a student of sociology who, along with two friends (Terry O. Nicholson and Jeff Margrave), forms an expedition party to explore an area of uncharted land where it is rumored lives a society consisting entirely of women. The three friends do not entirely believe the rumors because they are unable to think of a way how human reproduction could occur without males. The men speculate about what a society of women would be like, each guessing differently based on the stereotype of women which he holds most dear: Jeff regarding women as things to be served and protected; Terry viewing them as things to be conquered and won.
When the explorers reach their destination, they proceed with caution, hiding the biplane they arrive in, and trying to keep themselves hidden in the forests that border the land. They are quickly found by three young women who they realize are observing them from the treetops. After attempting to catch the girls with trickery, the men end up chasing the young women towards a town or village. The women outrun them easily and disappear among the houses, which, Van notes are exceptionally well made and attractive. After meeting the first inhabitants of this new land (which Van names Herland) the men proceed more cautiously, noting that the girls they met were strong, agile, and completely unafraid. Their caution is warranted because as the men enter the town where the girls disappeared, they become surrounded by a large group of women who march them towards an official looking building. The three men attempt an escape but are swiftly and easily overpowered by the large group of women and eventually anesthetized.
The men awake to find themselves held captive in a fortress-like building. They are given comfortable living accommodations, clean clothes, and food. The women assign each man a tutor who teaches the men their language. Van makes many notes about the new country and people, commenting that everything from their clothing to their furniture seems to be made with the twin ideals of pragmatism and aesthetics given equal consideration. The women themselves appear intelligent and astute, unafraid and patient, with a notable lack of temper and seemingly limitless understanding for their captives. The women are keen to learn about the world outside and question the men eagerly about all manner of things. Often Van finds himself having difficulty justifying the practices of his own society such as the milking of cows, the keeping of dogs as pets, and abortion, when faced with the apparent utopia the women have managed to build.
After being held captive for a number of months, the men break out of the fortress and escape cross-country to where they left their biplane. Finding the biplane sewn inside a large fabric covering, the men are unable to get away and are resignedly recaptured by the women. They are treated well nonetheless and soon learn that they will be given a freer rein when they have mastered the women's language and proved they can be trusted. Van remarks upon Terry's personal difficulty in dealing with the women who steadfastly refuse to conform to his expectations of how women should act, though Jeff seems perfectly enamored of the women and their kindness.
Van gradually finds out more information about the women's society, discovering that most of the men were killed 2,000 years ago when a volcanic eruption sealed off the only pass out of Herland. The remaining men were mostly slaves who killed the sons of their dead masters and the old women, intending to take over the land and the young women with it. The women fought back, however, killing the slaves. After a period of hopelessness at the impending end of their race, cut off from the rest of the world and without any men, one woman among the survivors became pregnant and bore a female child, and five more female children after. The five daughters of this woman also grew up to bear five daughters each. This process rapidly expanded their population and led to the exaltation of motherhood. Ever since that time the women had devoted themselves to improving their minds, working together and raising their children; the position of teacher being one of the most revered and respected positions in the land.
As the men are allowed more freedom, each strikes up a relationship with one of the women they had first seen upon their arrival, Van with the one called Ellador, Jeff with Celis, and Terry with Alima. Having had no men for 2,000 years the women apparently have no experience or cultural memory of romantic love or sexual intercourse. As such, the couples' budding relationships progress with some difficulty and much explanation. Terry in particular finds it hard to adjust to being in a relationship with a woman who is not a 'woman' in his terms. Eventually all three couples get 'married', although the women largely fail to see the point of such a thing and as they have no particular religion the ceremony is more pagan than Christian.
Their marriages cause the men much reflection; the women they married have no conception of what being a wife or being feminine entails (according to the outside world's views). Van admits finding it frustrating and hard sometimes, though he is in the end grateful for his wonderful friendship with Ellador and the intense love he feels for her. Terry is not so wise and out of frustration attempts to rape Alima. After being forcefully restrained and once again anesthetized, Terry stands trial before the women and is ordered to return to his homeland. The other men, while disapproving of Terry's actions, see them as impolite rather than criminal. Van explains to Ellador, "[Crime]'s a pretty hard word for it. After all, Alima was his wife, you know."
Van realizes that he must accompany Terry home in the biplane and Ellador will not let him leave without her. In the end, both Terry and Van leave Herland with promises not to reveal the utopia until Ellador has returned and such a plan has been fully discussed. Jeff however chooses to stay behind and live in Herland with his now pregnant wife, Celis. Van tries to prepare Ellador for returning to his world but feels much trepidation about what she will find there.