I think that is part of it, but I also think that he's just expecting a civil war to start. They say that liberals are weak and stupid publicly, but in private, they know it's exactly the opposite of truth. He fears the educated ppl coming together to overthrow him.