The novel is set amongst the villages of the Igbo people in British Nigeria during the 1920s. Ezeulu is the chief priest of the god Ulu, worshipped by the six villages of Umuaro. The book begins with Ezeulu and Umuaro fighting against a nearby village, Okperi. The conflict is abruptly resolved when T.K. Winterbottom, the British colonial overseer, intervenes.
After the conflict, a Christian missionary, John Goodcountry, arrives in Umuaro. Goodcountry begins to tell the villages tales of Nigerians in the Niger Delta who abandoned (and battled) their traditional "bad customs" in favor of Christianity.
Ezeulu is called away from his village by Winterbottom and is invited to become a part of the colonial administration, a policy known as indirect rule. Ezeulu refuses to be a "white man's chief" and is thrown in prison. In Umuaro, the people cannot harvest the yams until Ezeulu has called the New Yam Feast to give thanks to Ulu. When Ezeulu returns from prison, he refuses to call the feast despite being implored by other important men in the village to compromise. Ezeulu reasons to the people and to himself that it is not his will but Ulu's; Ezeulu believes himself to be half spirit and half man. The yams begin to rot in the field, and a famine ensues for which the village blames Ezeulu. Seeing this as an opportunity, John Goodcountry proposes that the village offer thanks to the Christian God instead so that they may harvest what remains of their crops with "immunity".
Many of the villagers have already lost their faith in Ezeulu. One of Ezeulu's sons dies during a traditional ceremony, and the village interprets this as a sign that Ulu has abandoned their priest. Rather than face another famine, the village converts to Christianity.
The title "Arrow of God" refers to Ezeulu's image of himself as an arrow in the bow of his god.