When Tom Long's brother Peter gets measles, Tom is sent to stay with his Uncle Alan and Aunt Gwen. They live in an upstairs flat of a big house with no garden, only a tiny yard for parking. The former grounds of the big house have been sold for building and are occupied by modern houses. The elderly and reclusive landlady, Mrs Bartholomew, lives above them. Because Tom may be infectious, he is not allowed out to play, and he feels lonely. Without exercise he lies awake after midnight, restless, when he hears the communal grandfather clock strangely strike 13. He gets up to investigate and discovers that the back door now opens on a large sunlit garden. However, when Tom checks the back door the following morning, the garden is no longer there.
Every night the clock strikes 13 and Tom returns to the Victorian era grounds. There he meets another lonely child, a girl called Hatty, and they become inseparable playmates. Tom sees the family occasionally, but only Hatty (and as is revealed later in the book, the gardener) sees him and the others believe she plays alone. Hatty is established to be an orphan sent to live with her aunt and three older male cousins after the death of her parents.
Tom writes daily accounts to his brother Peter, who follows the adventures during his recovery – and afterward, for Tom contrives to extend the stay with Aunt and Uncle. Gradually at first, Hatty grows up and passes Tom's age; he comes to realise that he is slipping to different points in the past. Finally she grows up at a faster rate, until she is an adult and is being courted by an acquaintance of hers who is nicknamed "Barty." At this stage in the book, the season in the old garden tends to be winter. Tom ingeniously obtains ice skates by having Hatty conceal her old pair in his room, where he subsequently finds them and joins her skating on the next night.
On the final night before Tom is due to go home, he goes downstairs to find the garden is not there. He frantically tries to find it, but crashes into a set of bins from the present–day courtyard, waking up several residents. He shouts Hatty's name in desperation, before his Uncle Alan finds him and puts the events down to Tom sleepwalking. The following morning, Mrs Bartholomew summons Tom to apologise, only to reveal herself as Hatty, having made the link when she heard him call her name. The events Tom experienced were real in Hatty's past; he has stepped into them by going into the garden at the times she dreamt of them. On the final night, she had instead been dreaming of her wedding with Barty.
After taking Tom home, Aunt Gwen comments on the strange way that Tom had said goodbye to Mrs Bartholomew when he left: he hugged her as if she were a little girl.
