Wealthy industrialist Phillip Colbert and his wife are in Sparta, Mississippi, to oversee the building of a factory. Late one night, police officer Sam Wood discovers Colbert's murdered body lying in the street. Wood finds Virgil Tibbs, a black man with a fat wallet, at the train station and arrests him. Police chief Bill Gillespie accuses him of murder and robbery, but soon learns Tibbs is a top homicide detective from Philadelphia, who was passing through town after visiting his mother. Tibbs wants to leave town on the next train, but his Chief in Philadelphia suggests he stay in Sparta to help Gillespie with the murder investigation. Though Gillespie, like many of Sparta's white residents, is racist, he and Tibbs reluctantly agree to work together.
A doctor estimates that Colbert had been dead for less than an hour when his body was found. Tibbs examines the body and concludes the murder happened much earlier, the killer was right-handed, and the victim had been killed elsewhere and moved to where Wood found his body.
Gillespie arrests another suspect, Harvey Oberst, who protests his innocence. The police plan to beat him to extract a confession, but Tibbs reveals Oberst is left-handed and has witnesses to confirm his alibi. Frustrated by the ineptitude of the police but impressed by Tibbs, Colbert's widow threatens to halt construction of the factory unless Tibbs leads the investigation, so the town's leading citizens are forced to comply with her demand.
Tibbs initially suspects the murderer is wealthy plantation owner Eric Endicott, a genteel racist and Sparta's most powerful citizen, who publicly opposed Colbert's new factory. When Tibbs begins interrogating him, Endicott slaps him, to which Tibbs responds by slapping him back. Afterwards, Endicott sends a gang of thugs after him. Gillespie rescues Tibbs and tells him to leave town to save himself, but Tibbs is determined to stay and solve the case.
Tibbs asks Officer Wood to re-trace his patrol car route during the night of the murder; Gillespie joins them. After questioning why Wood partially detours from his patrol route, Tibbs discovers that Wood enjoys passing by the house of 16-year-old Delores Purdy, who deliberately walks around nude with the lights on in an attempt to entice him, and that Wood changed his route to prevent Tibbs from seeing her. Gillespie learns that Wood made a $632 deposit to his bank account the day after the murder. He arrests Wood, despite Tibbs's protests that he is not the murderer. Tibbs tells Gillespie that the murder was committed at the site of the planned factory, which clears Wood because he could not have driven both his and Colbert's cars back into town. Later, Wood provides a credible account of where the money for his large deposit could have come from.
Delores' older brother Lloyd, a hostile racist, brings her to the police station to file statutory rape charges against Wood for getting her pregnant. When Tibbs insists on being present during Delores' questioning, Lloyd is offended that a black man is present during her interrogation and soon afterwards gathers a lynch mob to attack Tibbs.
Tibbs pressures illegal abortionist Mama Caleba to reveal that she is about to provide an abortion for Delores. When Delores arrives and sees Tibbs, she runs away. Tibbs follows Delores and confronts her armed boyfriend, Ralph Henshaw, a cook at a roadside diner. Purdy's mob arrives and holds Tibbs at gunpoint.
Tibbs tells Purdy to check Delores' purse for the $100 Ralph gave her for an abortion, which he got from killing and robbing Colbert. Lloyd realizes Tibbs is right when he opens the purse and finds the money. After Lloyd confronts Ralph for getting his sister pregnant, Ralph shoots Lloyd dead. Tibbs grabs Ralph's gun. Ralph is arrested and confesses to the killing of Colbert. He explains that after hitchhiking a ride with Colbert and asking him for a job, Ralph attacked him at the construction site of the new factory, intending only to knock Colbert unconscious and rob him, but accidentally killing him instead.
Tibbs arrives at the station to meet his train to return to Philadelphia, as Gillespie, having carried his suitcase, shakes Tibbs' hand and bids him farewell. In the final interaction between Gillespie and Tibbs, as the detective ascends the stairs onto the train, for one last time Gillespie calls out to him and sincerely tells Tibbs: "You take care, you hear?" After a moment of hesitation, Tibbs gives a warm smile in reply. Gillespie smiles back at Tibbs as he boards the train.
