What is the metaphorical value of the sweet potatoes in Invisible Man

It's interesting that the word "sweet", which is mentioned numerous times, is mixed up with the narrators nostalgia of home. The yams seem to be a metaphor for home, a place that the narrator misses or at least his childhood memories of it.