Believing that requires an optimistically expansive idea of an LLM's capabilities coupled with a regressive view of what a human mind is capable of, which probably explains why so many of them fall back on "What, so a human mind never makes mistakes?"
And then you've got the naive view, which is all the masses of people who know very little about LLMs but encounter them 1) labeled as "artificial intelligence", 2) promoted as a source of knowledge and helpful assistant, and 3) read output formulated to be confident, personable, and helpful...
...and are fully prepared to accept this thing that has been sold to them as a sci-fi computer mind and which is designed to interact with them as though it is such a thing, because it looks like it is doing what the packaging tells them it is doing. This is dangerous and it sucks a whole lot.
I've seen at least two separate cases of AI-instigated death this week; they didn't happen this week but they're in the news this week. I don't think these are the only ones and I am confident they will not be the last.
And in between those poles, you've got the "I Want To Believe (So I Do)" AI Truthers, who aren't the naive end-users who only that the company says it's an artificial intelligence and boy, it sure looks intelligent, doesn't it? They know better, but they want to believe.
So they invent the God of the Gaps, where all the things we (or they, personally) don't know about how LLMs do what they do and about how the human mind does what it does are stretched until they overlap, and you get "But how do we know it's not thinking?"
If doing statistical regression about word occurrences/correlations in a mind-bogglingly vast corpus of written works sometimes allows a program to produce results that seem unexpectedly life-like or eerily accurate... It's still just doing statistical regression about abstract tokens.
A quote of the first post raises a question: could I say definitively that doing sufficiently complex statistical operations on abstract tokens is not a form of cognition equally valid to human thought? My answer has two parts.
The first part begins: well, we could define cognition such that any process through which factors are weighed to arrive at a conclusion, in which case a pocket calculator is capable of cognition. So is a chain of dominoes set up to knock each other over. Or the operation of a pinball plunger. But.
Unless things go very wrong, the calculator will always arrive at the same conclusion. The dominoes are more variable because of the vagaries of macroscale physics, but there's a strong likelihood of a single conclusion (the last domino falls) and not much coherence among the alternatives.
I'm leery of rooting a definition of true/meaningful cognition in the realm of stochastic vs. determinist. When I'm anxious, I recite square numbers and the Fibonnaci sequence in my head, both of them from 0 to 144 (the symmetry soothes me). Sometimes, this is recall. Sometimes, it's calculation.