Developments before Darwin's theory
In later editions of the book, Darwin traced evolutionary ideas as far back as Aristotle; the text he cites is a summary by Aristotle of the ideas of the earlier Greek philosopher Empedocles. Early Christian Church Fathers and Medieval European scholars interpreted the Genesis creation narrative allegorically rather than as a literal historical account; organisms were described by their mythological and heraldic significance as well as by their physical form. Nature was widely believed to be unstable and capricious, with monstrous births from union between species, and spontaneous generation of life.
The Protestant Reformation inspired a literal interpretation of the Bible, with concepts of creation that conflicted with the findings of an emerging science seeking explanations congruent with the mechanical philosophy of René Descartes and the empiricism of the Baconian method. After the turmoil of the English Civil War, the Royal Society wanted to show that science did not threaten religious and political stability. John Ray developed an influential natural theology of rational order; in his taxonomy, species were static and fixed, their adaptation and complexity designed by God, and varieties showed minor differences caused by local conditions. In God's benevolent design, carnivores caused mercifully swift death, but the suffering caused by parasitism was a puzzling problem. The biological classification introduced by Carolus Linnaeus in 1735 also viewed species as fixed according to the divine plan. In 1766, Georges Buffon suggested that some similar species, such as horses and asses, or lions, tigers, and leopards, might be varieties descended from a common ancestor. The Ussher chronology of the 1650s had calculated creation at 4004 BC, but by the 1780s geologists assumed a much older world. Wernerians thought strata were deposits from shrinking seas, but James Hutton proposed a self-maintaining infinite cycle, anticipating uniformitarianism.
Charles Darwin's grandfather Erasmus Darwin outlined a hypothesis of transmutation of species in the 1790s, and Jean-Baptiste Lamarck published a more developed theory in 1809. Both envisaged that spontaneous generation produced simple forms of life that progressively developed greater complexity, adapting to the environment by inheriting changes in adults caused by use or disuse. This process was later called Lamarckism. Lamarck thought there was an inherent progressive tendency driving organisms continuously towards greater complexity, in parallel but separate lineages with no extinction. Geoffroy contended that embryonic development recapitulated transformations of organisms in past eras when the environment acted on embryos, and that animal structures were determined by a constant plan as demonstrated by homologies. Georges Cuvier strongly disputed such ideas, holding that unrelated, fixed species showed similarities that reflected a design for functional needs. His palæontological work in the 1790s had established the reality of extinction, which he explained by local catastrophes, followed by repopulation of the affected areas by other species.
In Britain, William Paley's Natural Theology saw adaptation as evidence of beneficial "design" by the Creator acting through natural laws. All naturalists in the two English universities (Oxford and Cambridge) were Church of England clergymen, and science became a search for these laws. Geologists adapted catastrophism to show repeated worldwide annihilation and creation of new fixed species adapted to a changed environment, initially identifying the most recent catastrophe as the biblical flood. Some anatomists such as Robert Grant were influenced by Lamarck and Geoffroy, but most naturalists regarded their ideas of transmutation as a threat to divinely appointed social order.
Inception of Darwin's theory
Darwin went to Edinburgh University in 1825 to study medicine. In his second year he neglected his medical studies for natural history and spent four months assisting Robert Grant's research into marine invertebrates. Grant revealed his enthusiasm for the transmutation of species, but Darwin rejected it. Starting in 1827, at Cambridge University, Darwin learnt science as natural theology from botanist John Stevens Henslow, and read Paley, John Herschel and Alexander von Humboldt. Filled with zeal for science, he studied catastrophist geology with Adam Sedgwick.
In December 1831, he joined the Beagle expedition as a gentleman naturalist and geologist. He read Charles Lyell's Principles of Geology and from the first stop ashore, at St. Jago, found Lyell's uniformitarianism a key to the geological history of landscapes. Darwin discovered fossils resembling huge armadillos, and noted the geographical distribution of modern species in hope of finding their "centre of creation". The three Fuegian missionaries the expedition returned to Tierra del Fuego were friendly and civilised, yet to Darwin their relatives on the island seemed "miserable, degraded savages", and he no longer saw an unbridgeable gap between humans and animals. As the Beagle neared England in 1836, he noted that species might not be fixed.
Richard Owen showed that fossils of extinct species Darwin found in South America were allied to living species on the same continent. In March 1837, ornithologist John Gould announced that Darwin's rhea was a separate species from the previously described rhea (though their territories overlapped), that mockingbirds collected on the Galápagos Islands represented three separate species each unique to a particular island, and that several distinct birds from those islands were all classified as finches. Darwin began speculating, in a series of notebooks, on the possibility that "one species does change into another" to explain these findings, and around July sketched a genealogical branching of a single evolutionary tree, discarding Lamarck's independent lineages progressing to higher forms. Unconventionally, Darwin asked questions of fancy pigeon and animal breeders as well as established scientists. At the zoo he had his first sight of an ape, and was profoundly impressed by how human the orangutan seemed.
In late September 1838, he started reading Thomas Malthus's An Essay on the Principle of Population with its statistical argument that human populations, if unrestrained, breed beyond their means and struggle to survive. Darwin related this to the struggle for existence among wildlife and botanist de Candolle's "warring of the species" in plants; he immediately envisioned "a force like a hundred thousand wedges" pushing well-adapted variations into "gaps in the economy of nature", so that the survivors would pass on their form and abilities, and unfavourable variations would be destroyed. By December 1838, he had noted a similarity between the act of breeders selecting traits and a Malthusian Nature selecting among variants thrown up by "chance" so that "every part of newly acquired structure is fully practical and perfected".
Darwin now had the framework of his theory of natural selection "by which to work", but he was fully occupied with his career as a geologist and held off writing a sketch of his theory until his book on The Structure and Distribution of Coral Reefs was completed in May 1842.
Darwin continued to research and extensively revise his theory while focusing on his main work of publishing the scientific results of the Beagle voyage. He tentatively wrote of his ideas to Lyell in January 1842; then in June he roughed out a 35-page "Pencil Sketch" of his theory. Darwin began correspondence about his theorising with the botanist Joseph Dalton Hooker in January 1844, and by July had rounded out his "sketch" into a 230-page "Essay", to be expanded with his research results and published if he died prematurely.
In November 1844, the anonymously published popular science book Vestiges of the Natural History of Creation, written by Scottish journalist Robert Chambers, widened public interest in the concept of transmutation of species. Vestiges used evidence from the fossil record and embryology to support the claim that living things had progressed from the simple to the more complex over time. But it proposed a linear progression rather than the branching common descent theory behind Darwin's work in progress, and it ignored adaptation. Darwin read it soon after publication, and scorned its amateurish geology and zoology, but he carefully reviewed his own arguments after leading scientists, including Adam Sedgwick, attacked its morality and scientific errors. Vestiges had significant influence on public opinion, and the intense debate helped to pave the way for the acceptance of the more scientifically sophisticated Origin by moving evolutionary speculation into the mainstream. While few naturalists were willing to consider transmutation, Herbert Spencer became an active proponent of Lamarckism and progressive development in the 1850s.
Hooker was persuaded to take away a copy of the "Essay" in January 1847, and eventually sent a page of notes giving Darwin much needed feedback. Reminded of his lack of expertise in taxonomy, Darwin began an eight-year study of barnacles, becoming the leading expert on their classification. Using his theory, he discovered homologies showing that slightly changed body parts served different functions to meet new conditions, and he found an intermediate stage in the evolution of distinct sexes.
Darwin's barnacle studies convinced him that variation arose constantly and not just in response to changed circumstances. In 1854, he completed the last part of his Beagle-related writing and began working full-time on evolution. His thinking changed from the view that species formed in isolated populations only, as on islands, to an emphasis on speciation without isolation; that is, he saw increasing specialisation within large stable populations as continuously exploiting new ecological niches. He conducted empirical research focusing on difficulties with his theory. He studied the developmental and anatomical differences between different breeds of many domestic animals, became actively involved in fancy pigeon breeding, and experimented (with the help of his son Francis) on ways that plant seeds and animals might disperse across oceans to colonise distant islands. By 1856, his theory was much more sophisticated, with a mass of supporting evidence.