It's just true. They're not going to "speak directly" because it's not a thing in the world in general society for neurotypical people the way you want it. I don't agree with your definition of ableism and I'm physically disabled, BTW. Learning how to say something and what to say is not beyond