(And not necessarily because they will hit a lot of people, but because they won't be able to discern what to do and will probably often just stop and sit there.)

Yup. Experienced it.

The opposite, actually! Waymo is going to be traveling Manhattan streets in a couple years; but I think it'll be much longer than that before you can buy a car that can take you from any randomly-picked US address to another without driving.

The problem with self-driving cars in Manhattan isn't going to be the "self-driving" part, it's the "car" part--they're just an inherently inefficient mode in a place where space is at a premium.