A group of children are playing an elimination game in the courtyard of an apartment building in Berlin using a chant about a murderer of children. A woman sets the table for dinner, waiting for her daughter to come home from school. A wanted poster warns of a serial killer preying on children, as anxious parents wait outside a school.
Little Elsie Beckmann leaves school, bouncing a ball on her way home. She is approached by Hans Beckert, who is whistling "In the Hall of the Mountain King" by Edvard Grieg. He offers to buy her a balloon from a blind street-vendor and walks and talks with her. Elsie's place at the dinner table remains empty, her ball is shown rolling away across a patch of grass and her balloon is lost in the telephone lines overhead.
In the wake of Elsie's disappearance, anxiety runs high among the public. Beckert sends an anonymous letter to the newspapers, taking credit for the murders and promising that he will commit others; the police extract clues from the letter, using the new techniques of fingerprinting and handwriting analysis. Under mounting pressure from city leaders, the police work around the clock. Inspector Karl Lohmann instructs his men to intensify their search and to check the records of recently released psychiatric patients, focusing on any with a history of violence against children. They stage frequent raids to question known criminals, disrupting underworld business so badly that Der Schränker (The Safecracker) calls a meeting of the city's crime lords. They decide to organize their own manhunt, using beggars to watch the children. Meanwhile, the police search Beckert's rented rooms, find evidence that he wrote the letter there, and lie in wait to arrest him.
Beckert sees a young girl in the reflection of a shop window and begins to follow her, but stops when the girl meets her mother. He encounters another girl and befriends her, but the blind vendor recognizes his whistling. The blind man tells one of his friends, who tails the killer with assistance from other beggars he alerts along the way. Afraid that Beckert will get away, one man chalks a large M (for Mörder, murderer in German) on his palm, pretends to trip, and bumps into Beckert, marking the back of his overcoat.
Once Beckert realizes that the beggars are following him, he hides inside a large office building just before the workers leave for the evening. The beggars call Der Schränker, who arrives at the building with a team of other criminals. They capture and torture one of the watchmen for information and, after capturing the other two, search the building and catch Beckert in the attic. When one of the watchmen trips the silent alarm, the criminals narrowly escape with their prisoner before the police arrive. Franz, one of the criminals, is left behind in the confusion and captured by the police; Lohmann tricks him into admitting that the gang only broke into the building to find Beckert and revealing where he will be taken.
The criminals drag Beckert to an abandoned distillery to face a kangaroo court. He finds a large, silent crowd awaiting him. Beckert is given a "lawyer", who gamely argues in his defense but fails to win any sympathy from the "jury". Beckert delivers an impassioned monologue, saying that he cannot control his homicidal urges, while the other criminals present break the law by choice. Beckert pleads to be handed over to the police, asking, "Who knows what it's like to be me?" His "lawyer" points out that the presiding "judge" is wanted on three counts of totschlag (manslaughter, a form of homicide in German law), and that it is unjust to execute an insane man. Just as the enraged mob is about to kill Beckert, the police arrive to arrest both him and the criminals.
As a panel of judges prepares to deliver a verdict at Beckert's real trial, the mothers of three of his victims weep in the gallery. Elsie's mother says that no sentence will bring the dead children back, and that "One has to keep closer watch over the children." The screen fades to black as she adds, "All of you."