Maybe you’re stringing words together based on the probability of those words appearing in that order in your training dataset with no understanding of their meaning or context, but I’m pretty confident that’s not how communication works for me and the human people I communicate with.
That's not how LLMs work, if you believe so. Their reasoning is based on neural nets that are modeling all kinds of concepts and relationships.
An actual expert elsewhere in this thread already told you you are wrong and Katie is correct. Keep being aggressively wrong if you want to, but I don’t have to agree or even keep talking to you. Have a nice day.
Who's that actual expert?
You know when I have two people who are experts on a subject and they disagree on a major concept in their field, I tend to go with the one who isn't literally trying to sell me a product with their viewpoint, Pekka.
Who are you talking about? What concept? Who's selling a product?
Your initial skeet reply here was quoting a skeet of yours that had a video. I believe that what she meant
While Aaronson qualifies as an expert, I don't know what product he's supposedly selling. He spent some time on academic leave working on AI safety research at OpenAI but he's hardly a salesman. AFAIK He's back at the University of Texas at Austin as the director of its Quantum Information Center.
Seriously? Are we talking about the same humans? Have you watched the news lately?
Yes? People being wrong and unethical does not negate what I said. Did you read it??
You are just stringing words together. Go back and read what you wrote and then consider it in the context of current events
If you can’t understand the difference between stringing words together based only on quantitative analysis of word order probability and stringing words together based on qualitative analysis of what those words mean and what your intent is, then you aren’t equipped to have this conversation.
That's an example of the former, I presume.
I'm not jumping into the full context of this back and forth, but I did want to point something out. LLMs aren't a simple Markov Chain. In the attention block of the neural network, it's not working with tokens (or words) directly. It's working with the "meaning" as embedded in vector space.
Only after it goes through the attention layers does it get decoded into the most likely token. It's an important distinction.
I would further highlight how those internal vectors can have thousands of dimensions (+superpositions), representing very deep multidimensional understanding of each token. All those dimensions are reduced into one (by separate transformation matrices) basically just for input/output with us.
As Ilya put it (paraphrasing) "To predict the next token it has to understand"
You're just stringing words together to insult me & kind of wasting my time. I'm sorry you can't apply critical thinking to what you're saying 🤷♂️
It’s genuinely concerning to me that you think I’m the one not engaging in critical thinking here.
You can't see it and I'm sorry for you
I mean, you can say "LLMs are limited" and that's true. Any tool has limitations But when you say "LLMs are limited and humans aren't" then you've taken a leap into la-la land
Think about this: what's the counter-point to what you said before? Are you *able* to consider that?