The story begins with the narrator calling "Halloa! Below there!" into a railway cutting. The signalman standing on the railway below does not look up, as the narrator expects, but rather turns about and stares into the railway tunnel that is his responsibility to monitor. The narrator calls down again and asks permission to descend. The signalman seems reluctant.
The railway hole is a cold, gloomy, and lonely place. The signalman still seems to be in fear of the narrator, who tries to put him at ease. The signalman feels that he had seen the narrator before, but the narrator assures him that this is impossible. Reassured, the signalman welcomes the newcomer into his little cabin, and the two men speak of the signalman's work. His labour consists of a dull monotonous routine, but the signalman feels he deserves nothing better, as he wasted his academic opportunities when he was young, although he has been spending his time during his shifts teaching himself mathematics and learning a foreign language (albeit with questionable pronunciation). The narrator describes that the signalman seems like a dutiful employee at all times, except when he twice looks at his signal bell when it is not ringing. There seems to be something troubling the signalman, but he will not speak of it. Before the narrator leaves, the signalman asks of him not to call for him when he's back on the top of the hill or when he sees him the following day.
The next day, as directed by the signalman, the narrator returns and does not call. The signalman tells the narrator that he will reveal his troubles. He is haunted by a recurring spirit which he has seen at the entrance to the tunnel on separate occasions, and, with each appearance, there has followed a tragedy. In the first instance, the signalman heard the same words which the narrator said and saw a figure with its left arm across its face, while waving the other in desperate warning; he questioned it, but it vanished. He then ran into the tunnel, but found no-one; a few hours later, there was a terrible train crash with many casualties. During its second appearance, the figure was silent, with both hands before the face in an attitude of mourning; then, a beautiful young woman died in a train passing through. Finally, the signalman admits that he has seen the spectre several times during the past week.
The narrator is sceptical about the supernatural, and suggests that the signalman is suffering from hallucinations. During their conversation, the signalman witnesses a ghost and hears his bell ring eerily, but the narrator sees and hears nothing. The signalman is sure that these supernatural incidents are presaging a third tragic event waiting to happen, and is sick with fear and frustration: he does not understand why he should be burdened with knowledge of an incipient tragedy when he, a minor railway functionary, has neither the authority nor the ability to prevent it. The narrator believes that his new friend's imagination has been overtaxed and suggests taking him to see a doctor.
The next day, the narrator visits the railway cutting again and sees a mysterious figure at the mouth of the tunnel. This figure is not a ghost, however; it is a man, one of a group of officials investigating an incident on the line. The narrator discovers that the signalman is dead, having been struck by an oncoming train. He had been standing on the line, looking intently at something, and failed to get out of the way. The driver of the train explains that he attempted to warn the signalman of his danger: as the train bore down on the signalman the driver called out to him, "Below there! Look out! For God’s sake, clear the way!" Moreover, the driver waved his arm in warning even as he covered his face to avoid seeing the train strike the hapless signalman. The narrator notes the significance of the similarity between the driver's words and actions and those of the spectre as the signalman had earlier described them, but leaves the nature of that significance to the reader.
