Immediately following the success of Some Like It Hot, Billy Wilder and I.A.L. Diamond wished to make another film with Jack Lemmon. Wilder had originally planned to cast Paul Douglas as Jeff Sheldrake; however, after he died unexpectedly, Fred MacMurray was cast.
The initial concept for the film came from Brief Encounter by Noël Coward, in which Celia Johnson has an affair with Trevor Howard in his friend's apartment. However, due to the Hays Production Code, Wilder was unable to make a film about adultery in the 1940s. Wilder and Diamond also based the film partially on a Hollywood scandal in which high-powered agent Jennings Lang was shot by producer Walter Wanger for having an affair with Wanger's wife, actress Joan Bennett. During the affair, Lang used a low-level employee's apartment. Another element of the plot was based on the experience of one of Diamond's friends, who returned home after breaking up with his girlfriend to find that she had committed suicide in his bed.
Although Wilder generally required his actors to adhere exactly to the script, he allowed Jack Lemmon to improvise in two scenes: in one scene he squirted a bottle of nose drops across the room, and in another he sang while making a meal of spaghetti (which he strains through the grid of a tennis racket). In another scene, where Lemmon was supposed to mime being punched, he failed to move correctly and was accidentally knocked down. Wilder chose to use the shot of the genuine punch in the film. Lemmon also caught a cold when one scene on a park bench was filmed in sub-zero weather.
Art director Alexandre Trauner used forced perspective to create the set of a large insurance company office. The set appeared to be a very long room full of desks and workers; however, successively smaller people and desks were placed to the back of the room, ending up with children. He designed the set of Baxter's apartment to appear smaller and shabbier than the spacious apartments that usually appeared in films of the day. He used items from thrift stores and even some of Wilder's own furniture for the set.
The film's title theme, written by Charles Williams and originally titled "Jealous Lover", was first heard in the 1949 film The Romantic Age. A recording by Ferrante & Teicher, released as "The Theme from The Apartment", reached #10 on the Billboard Hot 100 chart later in 1960.