I think the artificial distinction between the code and the output also raises another problem for this theory. If the code is expressive such that the government couldn't say "AI must output racial sluts," then liability for code that doesn't prevent recommending suicide is obviously a 1A issue.

tbh, that'd be some pretty surprising ai output

Slurs not sluts lol

I truly envy people whose rare typos only make their writing more interesting.

Though the latter would not be surprise using Grok

I think the product liability is more complicated than just the outputted content on its own. It’s the totality of the circumstances that breach a duty of care: anthropomorphizing the AI, representing that it is safe and expert for this purpose, the failure to sufficiently warn of risks of harm, etc

And for the totality of these circumstances, the speech content is unprotected because it is instrumental to the unlawful conduct, almost like a con or fraud. If it was content on its own (like a search result) then it’s different in kind — and it wouldn’t induce reliance in a reasonable person.

The illegality isn’t “the chatbot said he should kill himself and he did it.” The illegality is “the company offered an inherently dangerous product and failed to mitigate the reasonably foreseeable harm or adequately warn the user.” The speech is evidence of the illegality, not the whole of it.

But the reasonably foreseeable harm IS the content! You just can't separate them.

No, the reasonably foreseeable harm is the injury to the user. The content is the water that comes out of the kettle, and it’s scalding hot because the corp made a dangerous product. I’m not separating them; I’m saying the speech is an instrumentality of unlawful conduct.

So are all the woo woo books about how rocks can cure your cancer or whatever unprotected?

No, because the words on their own aren’t a dangerous product, as I said before. It’s the totality of the circumstances of the corp’s actions that make a dangerous product here, including their representations about what it can do.

To give another example, using autodialer technology is legal and in many cases very useful (school closure notifications). But when the same tech is used to perpetrate robocall scams, that is unlawful and not protected speech.

I think that's a completely artificial distinction. If you take away the content, there is no foreseeable harm.

No, that’s not correct. If the same system existed but didn’t output this specific content, then the foreseeable risk of harm exists, but in that scenario it just didn’t materialize. Eg, most users don’t encounter this, but some do. Liability is based on failure to adequately mitigate *risk*.

The code did not, independently of content, generate an unsafe condition.

And that's not the speech integral rule, that would be quite an expansion of it.

I just don't think twisting doctrine to assuage moral panic is going to lead us to good places. Yes, this was tragic. But we've seen before over and over that the rush to make sure the law allows "accountability" in response to emotionally difficult circumstances usually backfires.

It’s not twisting doctrine. Speech integral to unlawful conduct is not protected. That’s not limited to crimes. That’s why failure to warn claims aren’t blocked by the first amendment, among other types of consumer protection actions. A ton of privacy, civil rights, and fraud law relies on it.

Either way, I suspect we're entering the "do not attempt to stop chainsaw blade with genitals" era of AI.

If I join a book club whose ads states “We sell only books containing information about evidence-based medicine, supported by peer-reviewed research in accredited journals”, and they send me a book telling my I can cure cancer, heart disease, and athlete’s foot by eating prunes, is that not fraud?

You're looking at a false advertising claim at most there. And that's a commercial speech case.

The prune book’s speech is protected. That is not in dispute. The book club committed fraud by sending it to me. If not, why not? If I join the Cheese of the Month Club, and they send me a ham, and they claim “We don’t actually control what we ship. Sometimes, it’s cheese.” is that not fraud?

That's a twisting of the speech integral to criminal conduct rule

It’s not just for criminal conduct; it’s any unlawful conduct. See eg civil fraud.