Davidson wrote many stories for fiction magazines beginning in the 1950s, after publishing his first fiction in Commentary and other Jewish intellectual magazines.
Davidson was active in science fiction fandom from his teens. His best-known works are his novels about Vergil Magus, the magician that medieval legend made out of the Roman poet Virgil; the Peregrine novels, a comic view of Europe shortly after the fall of Rome; the Jack Limekiller stories, about a Canadian living in an imaginary Central American country modelled after Belize during the 1960s; and the stories of Dr. Eszterhazy, a sort of even more erudite Sherlock Holmesian figure living in the mythical Scythia-Pannonia-Transbalkania, the waning fourth-largest empire in Europe.
Lesser known and uncollected during his lifetime are his mystery stories, which were assembled after his death as The Investigations of Avram Davidson. These mystery stories frequently have a historical setting, and are intricately plotted. In addition, Davidson ghosted two Ellery Queen mysteries, And on the Eighth Day and The Fourth Side of the Triangle, and a true crime collection, Crimes and Chaos.
Other noteworthy works are his collaborations. In Joyleg, A Folly, written in collaboration with Ward Moore, a veteran of the American Revolutionary War (and of the Whiskey Rebellion) is found alive and very well in the Tennessee backwoods, having survived over the centuries by daily soaks in whisky of his own making to hilariously face the world of the 1960s. In Marco Polo and the Sleeping Beauty, co-written with Grania Davis, the background of Marco Polo's travels in the Mongol Empire is borrowed for an original story. After Davidson's death, Grania Davis also finished The Boss in the Wall, a claustrophobic horror novel that bears little resemblance to the work of any other writer.
Davidson also wrote dozens of short stories that defy classification, and the Adventures in Unhistory essays, which delve into puzzles such as the identity of Prester John and suggest solutions to them. His earlier historical essays were scrupulously researched, even when published by magazines just as happy to offer fiction as fact. Later essays were handicapped by a lack of resources in the libraries of the small towns where Davidson lived in the pre-Internet era, but are enlivened by the style and bold speculation.
Davidson's work is marked by a strong interest in history, with his plots often turning on what at first might seem like minor events. His characterization is also unusually in-depth for fantasy, and is often enriched by his ear for unusual accents.
Davidson's most obvious characteristics are his plotting and style. Very little may happen in a Davidson story, but he described it in detail. Hidden among the detail are facts or omissions that later become important to the outcome of the story. Especially in his later works, Davidson included elements that beginning writers are told to avoid, such as page-long sentences with half a dozen colons and semi-colons, or an apparently irrelevant digression in the opening pages of a story. He expects much from his readers, but delivers much to them.