The Phantom Tollbooth contains allusions to many works, including those loved by Juster in his own upbringing. Some of Juster's favorite books as a child, including The Wind in the Willows, had endpaper maps; Juster insisted on one, over Feiffer's opposition, going so far as to sketch one and require that his collaborator reproduce it in his own style. Juster was also inspired by his father Samuel's love of puns, with which the book is more than sprinkled. In his childhood, Juster spent much time listening to the radio. According to Juster, the need to envision the action when listening to radio serials helped inspire The Phantom Tollbooth, as well as yielding the character of Tock, based on sidekick Jim Fairfield from Jack Armstrong, the All-American Boy. Jim gave Tock his wisdom, courage, and adventurous spirit. As a child, Juster had synesthesia, and could only do arithmetic by making associations between numbers and colors. He remembered that the condition affected word associations. "One of the things I always did was think literally when I heard words. On the Lone Ranger [radio serial] they would say, 'Here come the Injuns!' and I always had an image of engines, of train engines."
"Oh dear, all those words again," thought Milo as he climbed into the wagon with Tock and the cabinet members. "How are you going to make it move? It doesn't have a—" "Be very quiet," advised the duke, "for it goes without saying." And, sure enough, as soon as they were all quite still, it began to move through the streets, and in a very short time they arrived at the royal palace.The Phantom Tollbooth (in Dictionopolis) 
Some of the incidents in the book stem from Juster's own past. In Digitopolis, the Numbers Mine, where gemlike numerals are dug for, recalled one of Juster's architecture professors at the University of Pennsylvania, who compared numbers and equations to jewels. The Marx Brothers films were a staple for Juster as a child and his father would quote lengthy passages from the movies; this inspired the unending series of straight-faced puns that fills the book.
Growing up in a Jewish-American household where the parents demanded high achievement, Juster was intimately familiar with expectations, though in his case many of his parents' hopes were centered on his older brother, an academic star. The Terrible Trivium, the well-dressed, polite demon who sets the questers to mindless tasks, was Juster's way of representing his own tendency to avoid what he should be doing in favor of a more congenial occupation, such as his evasion of the grant project to write The Phantom Tollbooth. Juster drew on Feiffer's life experiences as well; the Whether Man's adage "Expect everything, I always say, and the unexpected never happens" was a favorite of the cartoonist's mother.
Juster had not read Alice's Adventures in Wonderland when he wrote The Phantom Tollbooth, but the two books, each about a bored child plunged into a world of absurd logic, have repeatedly been compared. According to Daniel Hahn in his 2012 article on the Juster book, "Alice is clearly Milo's closest literary kin". Milo's conversation with the Whether Man, which leaves him no more comprehending than when he came, recalls that of Alice with the Cheshire Cat. The questions of authority (something omnipresent for a child) and of justice run through both books; the Queen of Hearts' arbitrary justice is echoed, though with less violence, by Officer Shrift. Alice's sovereigns, representing the authority figures of Victorian childhood life, rule absolutely (though not necessarily effectively); a child of the post-World War II world, Milo journeys through a more bureaucratic realm. His quest is far more purposeful than the frustrating journey Alice experiences, and the outcome differs as well—Milo restores his kingdom while Alice overturns hers. Carroll leaves us uncertain if Alice has learned anything from her adventures, but Juster makes it clear that Milo has acquired tools he will need to find his way through life.