A group of children are playing an elimination game in the courtyard of an apartment building in Berlin using a chant about a murderer of children. A woman sets the table for dinner, waiting for her daughter to come home from school. A wanted poster warns of a serial killer preying on children, as anxious parents wait outside a school.
Little Elsie Beckmann leaves school, bouncing a ball on her way home. She is approached by Hans Beckert, who is whistling "In the Hall of the Mountain King" by Edvard Grieg. He offers to buy her a balloon from a blind street-vendor and walks and talks with her. Elsie's place at the dinner table remains empty, her ball is shown rolling away across a patch of grass and her balloon is lost in the telephone lines overhead.
In the wake of Elsie's death, Beckert sends an angry letter about his crimes to the newspapers, from which the police extract clues using the new techniques of fingerprinting and handwriting analysis. Under mounting pressure from city leaders, the police work around the clock. Inspector Karl Lohmann instructs his men to intensify their search and to check the records of recently released psychiatric patients, to look for those with a history of violence against children. They stage frequent raids to question known criminals, disrupting underworld business so badly that Der Schränker (The Safecracker) calls a meeting of the city's crime lords. They decide to organize their own manhunt, using beggars to watch the children.
The police discover two clues corresponding to the killer's letter in Beckert's rented rooms. They wait there to arrest him.
Beckert sees a young girl in the reflection of a shop window. Following her, he is thwarted when the girl meets her mother. When he encounters another young girl, he succeeds in befriending her but the blind beggar recognizes his whistling. The blind man tells one of his friends, who tails the killer with assistance from other beggars he alerts along the way. Afraid of losing him, one young man chalks a large M (for Mörder, murderer in German) on his hand, pretends to trip and bumps into Beckert, marking the back of his clothing.
The beggars close in. When Beckert realizes he is being followed, he hides inside a large office building just before the workers leave for the evening. The beggars call Der Schränker and a team of criminals arrives. They tie up and torture a guard for information and after capturing the remaining watchmen, they search the building from coal cellar to attic, finally catching Beckert. When a watchman manages to trip the silent alarm, the crooks narrowly escape with their prisoner before the police arrive. One man is captured and tricked into revealing the purpose of the break-in (nothing was stolen) and where Beckert would be taken.
The criminals drag Beckert to an abandoned distillery to face a kangaroo court. He finds a large, silent crowd awaiting him. Beckert is given a "lawyer", who gamely argues in his defense but fails to win any sympathy from the "jury". Beckert delivers an impassioned monologue, saying that he cannot control his homicidal urges, while the other criminals present break the law by choice. Beckert pleads to be handed over to the police, asking, "Who knows what it's like to be me?" His "lawyer" points out that the presiding "judge" is wanted on three counts of totschlag (manslaughter, a form of homicide in German law), and that it is unjust to execute an insane man. Just as the enraged mob is about to kill him, the police arrive to arrest both Beckert and the criminals.
As the real trial passes, five judges prepare to pass judgement on Beckert. Before the sentence is announced, the shot cuts to three of the victims' mothers crying. Elsie's mother says no sentence would bring back the dead children and "One has to keep closer watch over the children"; the screen goes black as she adds, "All of you".