Davidson wrote many stories for fiction magazines beginning in the 1950s, after publishing his first fiction in Commentary and other Jewish intellectual magazines.
Davidson was active in science fiction fandom from his teens. His best-known works are his novels about Vergil Magus, the magician that medieval legend made out of the Roman poet Virgil; the Peregrine novels, a comic view of Europe shortly after the fall of Rome; the Jack Limekiller stories, about a Canadian living in an imaginary Central American country modelled after Belize during the 1960s; and the stories of Dr. Eszterhazy, a sort of even more erudite Sherlock Holmesian figure living in the mythical Scythia-Pannonia-Transbalkania, the waning fourth-largest empire in Europe.
Lesser known and uncollected during his lifetime are his mystery stories, which were assembled after his death as The Investigations of Avram Davidson. These mystery stories frequently have a historical setting, and are intricately plotted. In addition, Davidson ghosted two Ellery Queen mysteries, And on the Eighth Day and The Fourth Side of the Triangle, and a true crime collection, Crimes and Chaos.
Other noteworthy works are his collaborations. In Joyleg, A Folly, written in collaboration with Ward Moore, a veteran of the American Revolutionary War (and of the Whiskey Rebellion) is found alive and very well in the Tennessee backwoods, having survived over the centuries by daily soaks in whisky of his own making to hilariously face the world of the 1960s. In Marco Polo and the Sleeping Beauty, co-written with Grania Davis, the background of Marco Polo's travels in the Mongol Empire is borrowed for an original story. After Davidson's death, Grania Davis also finished The Boss in the Wall, a claustrophobic horror novel that bears little resemblance to the work of any other writer.
Davidson also wrote dozens of short stories that defy classification, and the Adventures in Unhistory essays, which delve into puzzles such as the identity of Prester John and suggest solutions to them. His earlier historical essays were scrupulously researched, even when published by magazines just as happy to offer fiction as fact. Later essays were handicapped by a lack of resources in the libraries of the small towns where Davidson lived in the pre-Internet era, but are enlivened by the style and bold speculation.
Davidson's work is marked by a strong interest in history, with his plots often turning on what at first might seem like minor events. His characterization is also unusually in-depth for fantasy, and is often enriched by his ear for unusual accents and his apparently endless ability to give each character his or her own characteristics of speech.
However, Davidson's most obvious characteristics are his plotting and style. Very little may happen in a Davidson story, but he enjoyed describing it in enormous detail. Hidden among the detail are facts or omissions that at the end of the story later prove to be the pebbles that start avalanches of major consequences. Especially in his later works, Davidson came to delight in deliberately including many elements that beginning writers are told to avoid, such as page-long sentences with half a dozen colons and semi-colons, or an irrelevant digression in the opening pages of a story. These touches often succeed through their sheer boldness, as well as the comedy that runs through many of his most ambitious works. In general, Davidson's attitude to his readers is similar to that of Nineteenth Century authors: He assumes that his readers are there to be amused, and will follow him wherever he happens to go.