In the background to the story in the book, an ancient and unseen alien race uses a device with the appearance of a large crystalline monolith to investigate worlds all across the galaxy and, if possible, to encourage the development of intelligent life. The book shows one such monolith appearing in ancient Africa, 3 million years B.C. (in the movie, 4 million years), where it inspires a starving group of hominids to develop tools. The ape-men use their tools to kill animals and eat meat, ending their starvation. They then use the tools to kill a leopard preying on them; the next day, the main ape character, Moon-Watcher, uses a club to kill the leader of a rival tribe. The book suggests that the monolith was instrumental in awakening intelligence.
The book then shows the year C.E. 1999, detailing Dr. Heywood Floyd's travel to Clavius Base on the Moon. Upon his arrival, Floyd attends a meeting, where a lead scientist explains that they have found a magnetic disturbance in Tycho, one of the Moon's craters, designated Tycho Magnetic Anomaly One, or TMA-1. An excavation of the area has revealed a large black slab, precisely fashioned to a ratio of exactly 1:4:9, or 1²:2²:3², and therefore believed the work of intelligence. Floyd and a team of scientists travel across the Moon to view TMA-1, and arrive as sunlight falls upon it for the first time in three million years. It then sends a piercing radio transmission to one of the moons of Saturn, Japetus (Iapetus), where an expedition is then planned to investigate.
The book then shows the Discovery One mission to Saturn, whereof Dr. David Bowman and Dr. Francis Poole are the only conscious human beings aboard, while their three colleagues are in suspended animation, to be awakened near Saturn. The HAL 9000, an artificially intelligent computer, addressed as "HAL", maintains the ship. While Poole is receiving a birthday message from his family on Earth, HAL tells Bowman that the AE-35 unit of the ship, keeping its communication active, is going to malfunction. Poole takes one of the extra-vehicular pods and swaps the AE-35 unit; but when Bowman conducts tests on the removed AE-35 unit, he determines that there was never anything wrong with it. Poole and Bowman become suspicious at HAL's refusal to admit that his diagnosis was mistaken; HAL then claims that the replacement AE-35 unit will fail. In communicating with Earth, Poole and Bowman are directed to disconnect HAL for analysis. These instructions are interrupted as the signal is broken, and HAL informs them that the AE-35 unit has malfunctioned.
As Poole is removing the unit he is killed when his spacesuit is torn, exposing him to the vacuum of space. Bowman, uncertain of HAL's role therein, decides to wake the other three astronauts, and therefore quarrels with HAL, with HAL refusing to obey his orders. Bowman threatens to disconnect him if his orders are not obeyed, and HAL relents. As Bowman begins to awaken his colleagues, he hears HAL open both airlocks into space, releasing the ship's internal atmosphere. From a sealed emergency shelter, Bowman gains a spacesuit and re-enters the ship, where he shuts down HAL's consciousness, leaving intact only his autonomic functions, and manually re-establishes contact with Earth. He then learns that his mission is to explore Iapetus,in the hope of contacting the society that buried the monolith on the Moon. Bowman learns that HAL had begun to feel guilty at keeping the purpose of the mission from him and Poole, against his stated mission of gathering information and reporting it fully; and when threatened with disconnection, he panicked and defended himself out of a belief that his very existence was at stake, having no concept of sleep.
Bowman spends months on the ship alone, slowly approaching Iapetus. During his approach, he gradually notices a small black spot on the surface of Iapetus, and later finds it identical in shape to TMA-1, only much larger. The scientists on Earth name this monolith "TMA-2", which Bowman identifies as a double misnomer because it is not in the Tycho crater and gives off no magnetic anomaly. When Bowman approaches the monolith, it opens and pulls in Bowman's pod. Before he vanishes, Mission Control hears him proclaim: "The thing's hollow — it goes on forever — and — oh my God! — it's full of stars!"
Bowman is transported via the monolith to an unknown star system, through a large interstellar switching station, and sees other species' spaceships going on other routes. Bowman is given a wide variety of sights, from the wreckage of ancient civilizations to what appear to be life-forms, living on the surfaces of a binary star system's planet. He is brought to what appears a pleasant hotel suite, carefully designed to make him feel at ease, and falls asleep, whereupon his mind and memories are drained from his body, and he becomes an immortal 'Star Child', that can live and travel in space. The Star Child then returns to Earth, where he detonates an orbiting nuclear warhead. This is not discussed again until the sequel to the book, 2010: Odyssey Two.