The plaintiffs are asserting products liability, negligence, wrongful death, and finally they’re asserting a violation of California Unfair Competition Law The UCL theory is GPT 4o violated Cal. criminal code by deliberately aiding & encouraging suicide, was practicing unlicensed psychotherapy,
This is horrific.
Das Geschäft mit der Abhängigkeit www.nbcnews.com/tech/tech-ne...
Menticide only other thing coming to mind as a claim this moment but that is irking me and feel like something else big and possible very important is eluding me.
The suggestion that it is providing unlicensed medical/psychological care is exactly right. We wouldn't allow a human to keep doing this. Why allow a computer?
If this were a human they would have been arrested.
The theory would have to be (... wouldn't it?) that OpenAI did those things, not that GPT did. I don't see how a non-person gets liability attached here. (But, I'm not a law talking person!)
It's the same way that Boeing is liable when their plane's autopilot decides to fly directly into the ground. If you design and sell a product, you're legally responsible for the dangers associated with the design.
It's different when the product is speech, but even if the product liability claim fails because of 1A issues, unlicensed practice of medicine is likely to stick and since OpenAI is ultimately responsible for ChatGPT's output they would be liable.
There's speech occurring here, and it has 1A protections, but since ChatGPT isn't a person that speech has to belong to either OpenAI or the end user, and in this case I would say OpenAI is the responsible party for the model's output.
as far as i know, based on initial motions for dismissal (due to 1A protections) by the defendants in Garcia v Character Tech (similar case), the judge dismissed the motion claiming there are no 1A protections because LLM output is not considered speech (as of now)
![Accordingly, the Court is not prepared to hold that Character A.I.’s output is speech. See W.W. v. Orlando Health, Inc., No. 6:24-cv-1068-JSS-RMN, 2025 WL 722892, at *7 (M.D. Fla. Mar. 6, 2025) (“[G]iven the lack of binding authority and the split in persuasive authority on this issue, the court will not dismiss Plaintiff’s Case 6:24-cv-01903-ACC-UAM Document 115 Filed 05/21/25 Page 31 of 49 PageID 908 - 32 - allegations while her case is in its infancy.”) (citing Sartori v. Schrodt, No. 3:18-cv- 204-RV/CJK, 2018 WL 11209992, at *2 (N.D. Fla. May 22, 2018); Palmyra Park Hosp., Inc. v. Phoebe Putney Mem’l Hosp., Inc., No. 1:08-cv-102 (WLS), 2009 WL 10673436, at *6 (M.D. Ga. Mar. 31, 2009)).](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:os4mjp4b5ly7grrpg24rqe42/bafkreihusuj72bb5rchaxt6vzvga2hlqdcsxj6erbaa4fqzi6rioduqdgu@jpeg)
What the court is saying here is that no higher court has held that it *is* speech and thus, accepting every argument the plaintiffs make as true(and they argue it isn't), it cannot dismiss the case before discovery.
yeah that's what i said
Also reading the full paragraph it seems the defendants didn't even make an argument, they just threw 1A out there without doing any work.
And I think that was intentional, because the most compelling arguments for the LLM output being speech would make the defendants clearly liable for that speech, even if gets them out of *some* of the claims on 1A grounds.
lots of very damning and deep analysis on why google (infrastructure and model supplier) is culpable in aiding wrongful death here www.courtlistener.com/docket/69300...
Yeah, they couldn't successfully sue for ChatGPT being offensive or something, but there's plenty of forms of speech that aren't 1A covered, especially in civil court.
And especially around medical and legal advice.
Got it, that makes sense.
"GPT 4o violated" implies "by design, GPT 4o violated", and therefore the corporate entity that approved the design is liable.
"foreseeable negligence" at the very least
Got it, thanks.
The "practicing unlicensed psychotherapy" part needs to be enforced in a big way. Insurance companies and startups are champing at the bit to force people into "AI therapists."
Also, it's an avenue significantly less likely to run into the 1st Amendment during enforcement. Product liability probably isn't going to stick here but unlicensed practice of medicine probably will.
From your mouth to the regulatory boards' ears. (Or rather, state legislatures' more likely.) (This is exactly what the "no regulations on AI for 10 years" clause in the BBB was trying to prevent.)
I wonder how the public and professional liability underwriters are feeling about that
Well AI Chatbots don't take out professional liability policies so no problem for them there!
😬 The Air Canada example might be worth attending to there
Work at a school where they introduced an "AI backed" peer for students to talk to. Happily, as far as I know, nobody has used it.
They're talking to somebody's chatbot, apparently. In shocking numbers.
They've always ran on "Move fast and break things". That was fine when they did trivial little websites where people could upload Pictures. Now they're breaking people.
disrupting the suicide forum industry by ingesting suicide how-to forums and then regurgitating that info as a pretend therapist robot
"who could have forseen this happening" other than basically anyone who said "this is going to happen"
I've just finished listening to Careless People on Audible.... one of those trivia websites (loss of l possibly deliberate) and you quote their mantra got awfully bad terribly quickly
The ones you could upload pics to broke enough people this is even worse
Absolutely not something to base the business off of. I can maybe whip up a heavy guardrail system to help tide someone over, but only as a free "self help" tool.
I use tele-therapy and it violates the rules if I'm in a different *state* as my therapist and yet these assholes want to have chatbots that work nationwide? 🙄😤
No license, no state licensing laws violated!
and that it violated Cal. health code by providing mental health services without the involvement of a guardian For the UCL violation they’re asking for an injunction requiring various safety features & warnings, parental controls, destruction of the model the decedent used, among other things
Good!
TBH I think this falls under "mental health DISservices" more than "services"
Can you give us the case name and number? I’m curious
I don’t see it docketed yet, but it’s in California Superior Court, San Francisco County, should be captioned as Raine v. OpenAI bsky.app/profile/saba...
Interesting that it’s Jay Edelson representing the family.
he's been critical of openai, I'm not surprised
Jay knows his shit.
Hey, Congress. Not such a good idea to ban the regulation of AI for a decade, as the House attempted. This is awful.
Did they only attempt, or did that pass? I'm having trouble keeping track.
As far as I can tell, it didn't pass. The provision on prohibition of state-level AI regulation in the Big Beautiful Bill Act got struck out by the Senate. So it was left out when the bill was reconciled with the House version and that version went on to pass into law.
Thank you!
📌
LLM guardrails will never work and the product can never become what the industry is claiming it will, but reading all the user analytics they had... just wow. How did their auto-moderation fail to kick in?
It will still download in a Zip file if you use the big blue “Download files” button.
there have been articles about this for a year and they haven't corrected the problem. I think this is the first person who has been killed by ChatGPT that we know of but it is NOT The first person who was told to end their lives.
Garcia v. Character Technologies. Not ChatGPT, but it’s a horrendously similar case with the same outcome for the victim.
Didn’t this also happen to someone in, I think Germany? I feel this has the tipping point action of showing how much at scale harm, it is actually delivering.
