She was born Lula Carson Smith in Columbus, Georgia in 1917. Her mother’s grandfather was a plantation owner and Confederate war hero. Her father was a watchmaker and jeweler of French Huguenot descent. From the age of ten she took piano lessons; when she was fifteen her father gave her a typewriter to encourage her story writing.
Smith graduated from Columbus High School. In September 1934, at age 17, she left home on a steamship bound for New York City, planning to study piano at the Juilliard School of Music. After falling ill with rheumatic fever she returned to Columbus to recuperate, and she changed her mind about studying music. Returning to New York she worked in menial jobs while pursuing a writing career; she attended night classes at Columbia University and studied creative writing under the Texas writer Dorothy Scarborough and with Sylvia Chatfield Bates at Washington Square College of New York University. In 1936 she published her first work. "Wunderkind", an autobiographical piece that Bates admired, depicted a music prodigy's adolescent insecurity and losses. It first appeared in Story magazine and is collected in The Ballad of the Sad Cafe.