I believe these models (and specifically ChatGPT) should be taken off the market. This study, published on Tuesday (the same day that Adam Raine's suicide was publicized) found that AI bots respond inadequately to users' expression of moderate suicidal risk.
A study of this sort cannot evaluate the intensity of responses over time, as in Adam Raine's case. I find it unconscionable that chatbots have been promoted as therapeutic agents - they are utterly unregulated, with no provision to monitor users' experiences.
I should clarify that by “monitor,” above, I mean “get feedback when the product is malfunctioning” (eg, reinforcing suicidal plans or giving other unacceptable, life-endangering responses).
(And I agree with Brian that OpenAI needs to be named and blamed specifically.)