why is higgins upset by eliza dialect?

Higgins is a scholar of language and various dialects. While he's fascinated by her dialect, it's low class character prohibits him from accepting her as a close companion, which she becomes even as he doesn't realize it. He wants her to improve it, both for his own success story and because he is growing closer to her but can't truly accept it through the class divisions epitomized in her harsh dialect.