People who used to be challenged by friends, by society, by communities will pick this AI as a refuge. But this AI is not driven by anything, it does not care about anything at all and will never challenge anyone to grow in any way. It's like we are placing rubber balls on every intellectual edge.
This thing will drown out real human screams and desires and replace them with the statistical average of yesterday. Why is this the scariest version of the singularity I can think of ? I would have preferred an angry HALL that questions me endlessly over a trashcan yes-man spawn machine. Honestly.