Suppose I created the program I described, and I was imprisoned because my program printed out "Trump is an idiot." You don't think that would constitute a violation of my first amendment rights? I think it is, also if it printed out "Trump is brilliant" (something I don't believe).
I think that for ChatGPT's speech to be recognized as Sam Altman's or whoever's would be a pretty catastrophic result for him because people would easily be able to get it to say stuff that wasn't protected and get him in trouble for it; such a ruling could well be game over for chatbots