According to Scorsese, it was Brian De Palma who introduced him to Schrader. In Scorsese on Scorsese, the director talks about how much of the film arose from his feeling that movies are like dreams or drug-induced reveries. He admits attempting to incubate within the viewer the feeling of being in a limbo state somewhere between sleeping and waking. He calls Travis an "avenging angel" floating through the streets of a New York City intended to represent all cities everywhere. Scorsese calls attention to improvisation in the film, such as in the scene between De Niro and Cybill Shepherd in the coffee shop. The director also cites Alfred Hitchcock's The Wrong Man and Jack Hazan's A Bigger Splash as inspirations for his camerawork in the movie.
In Scorsese on Scorsese, the director mentions the religious symbolism in the story, comparing Bickle to a saint who wants to cleanse or purge both his mind and his body of weakness. Bickle attempts to kill himself near the end of the movie as a tribute to the samurai's "death with honour" principle.
When Travis meets Betsy to join him for coffee and pie, she is reminded of a line in Kris Kristofferson's song "The Pilgrim, Chapter 33": "He's a prophet and a pusher, partly truth, partly fiction—a walking contradiction." On their date, Bickle takes her to see a Swedish sex education film, which is in fact the American sexplotation film Sexual Freedom in Denmark with added Swedish sound.
Shot during a New York summer heat wave and garbage strike, Taxi Driver came into conflict with the MPAA for its violence (Scorsese de-saturated the color in the final shoot-out, and the film got an R rating). To achieve the atmospheric scenes in Bickle's cab, the sound men would get in the trunk and Scorsese and his cinematographer, Michael Chapman, would ensconce themselves on the back seat floor and use available light to shoot.
In writing the script, Schrader was inspired by the diaries of Arthur Bremer (who shot presidential candidate George Wallace in 1972) and Fyodor Dostoyevsky's Notes from Underground. The writer also used himself as inspiration; prior to writing the screenplay, Schrader was in a lonely and alienated position, much like Bickle is. Following a divorce and a breakup with a live-in girlfriend, he spent a few weeks living in his car. He wrote the script in under a month while staying in his former girlfriend's apartment while she was away.
Schrader decided to make Bickle a Vietnam vet because the national trauma of the war seemed to blend perfectly with Bickle's paranoid psychosis, making his experiences after the war more intense and threatening. Thus, Bickle chooses to drive his taxi anywhere in the city as a way to feed his hate.
While preparing for his role as Bickle, De Niro was filming Bernardo Bertolucci's 1900 in Italy. According to Boyle, he would "finish shooting on a Friday in Rome ... get on a plane ... [and] fly to New York." De Niro obtained a cab driver's license, and when on break would pick up a cab and drive around New York for a couple of weeks, before returning to Rome to resume filming 1900. De Niro apparently lost 35 pounds and listened repeatedly to a taped reading of the diaries of Arthur Bremer. When he had time off from shooting 1900, De Niro visited an army base in Northern Italy and tape-recorded soldiers from the Midwestern United States, whose accents he thought might be appropriate for Travis's character.
When Bickle decides to assassinate Senator Palantine, he cuts his hair into a Mohawk. This detail was suggested by actor Victor Magnotta, a friend of Scorsese's who had a small role as a Secret Service agent and who had served in Vietnam. Scorsese later noted, "Magnotta had talked about certain types of soldiers going into the jungle. They cut their hair in a certain way; looked like a Mohawk ... and you knew that was a special situation, a commando kind of situation, and people gave them wide berths ... we thought it was a good idea."
Jodie Foster was not the first choice to play Iris. Scorsese considered Melanie Griffith, Linda Blair, Bo Derek, and Carrie Fisher for the role. A newcomer, Mariel Hemingway, auditioned for the role but turned it down due to pressure from her family. After the other actresses turned down the role as well, Foster, an experienced child actor, was chosen.
In the original draft, Schrader had written the role of Sport as a black man. There were also additions of other negative black roles. Scorsese believed that this would give the film an overly racist subtext, so they were changed to white roles. The Terminal Bar was featured in a scene in the film.
Schrader originally set the film in Los Angeles, but it was moved to New York City because taxicabs were much more prevalent there than in Los Angeles.
Taking place in an actual apartment, the tracking shot over the murder scene at the end took three months of preparation just because the production team had to cut through the ceiling in order to get it right.