It's not the diversity in the cast that is the problem. It is when the PR for your show/movie focuses solely on the diversity of the cast instead of what the show/movie brings to the lore. When before a movie/show comes out, you say "If you don't like this, then you're racist/bigot/etc."