The Ballad of the Sad Cafe opens in a small, isolated Georgia town. The story introduces Miss Amelia Evans, a strong character of both body and mind, who is approached by a hunchbacked man with only a suitcase in hand who claims to be of kin.
When Miss Amelia, whom the townspeople see as a calculating woman who never acts without reason, takes the stranger into her home, rumors begin to circulate that Miss Amelia has only done so to take what the hunchback had in his suitcase. When the rumors hit their peak, a group of eight men come to her store, sitting outside on the steps for the day and waiting to see if something would happen.
Finally, they enter the store all at once and are stunned to see that the hunchback is actually alive and well. With everyone gathered inside, Miss Amelia brings out some liquor and crackers in hospitality, which further shocks the men, as they have never witnessed Miss Amelia be hospitable enough to allow drinking inside her home. This is essentially the beginning of the café. Miss Amelia and the hunchback, Cousin Lymon, unintentionally create a new tradition for the town, and the people gather inside at the café on Sunday evenings often until midnight.
It is apparent, though surprising, to the townspeople that Miss Amelia has fallen in love with Cousin Lymon, and has begun to change slightly as time progresses. When the townspeople see this, they relate it to another odd incident in which Miss Amelia was also involved: the issue of her ten-day marriage.
Miss Amelia had been married to a man named Marvin Macy, who was a vicious and cruel character before meeting and falling in love with her. He changed his ways and became good-natured, but reverted to his old self when his love was rejected after a failed ten-day marriage in which he gave up everything he possessed in hopes of having her return his affections. He broke out into a rage, committing a string of felonies before being caught and locked up in the state penitentiary.
When he was released, he returned to the town with the full intention of ruining Miss Amelia's life the way she ruined his. Upon his return, he takes advantage of Cousin Lymon's admiration for him, as he views Macy as a true man, and uses him to crush Miss Amelia's heart. Macy and Miss Amelia engage in a physical fight, and just as Miss Amelia is about to take the upper hand, Lymon jumps her from behind allowing Macy to prevail. Macy and Cousin Lymon ransack the café, break her still, steal her curios and money, and disappear from town leaving Miss Amelia alone to herself.
The novella ends with The Twelve Mortal Men, which is a brief passage of twelve men in a chain-gang, whose actions outline that of what happened in the lonely rural town, and highlights the themes of loneliness and isolation.