We burn the planet for this ish?

But has anybody asked the obvious follow up question?

We should definitely let AI do air traffic control!

So I personally am against LLMs. But I tested this and did not find the same result. Just for fact checking.

LLMs make a statistical guess each time. They don’t develop a correct answer and remember it for later use. You can literally ask it the same question ten times and get ten different answers.

Inconsistency is indeed the problem. I asked gpt 4o the same question and got a different response

😆

Which iteration of AI will figure out how to count to 2? Nothing a few billion lit on fire can’t figure out, ….maybe.

Last line doing a lot of work in the AI response, lol

Complete Blubbery

v5 is a hybrid of multiple models, and will use the one that matches the perceived complexity of your question. So we're back to the "think very hard about this" world of prompt engineering.

See, the words LLMs use have no meaning. Their place is determined by probabilities, not meaning. So LLMs have no idea what "middle" means in this context. Something makes it decide 3 (which has no meaning) would be right if you used "middle" (middle is often used to describe one element of three).

That's addressed to the initial attempt. But a similar analysis would apply to the word "last" and maybe other in the second one. All those words often go together but what so?

Claude got it!

The fact that it broke it down like that makes me think Anthropic specifically included training data to address this problem lol

MLLMLs

Or this is a viral marketing campaign to get people to go try ChatGPT for themselves.

ChatGPT breaks text into tokens—small words or parts of words—which it codes as numbers. So “blueberry” gets broken into “blue” and “berry” which are coded as 18789 and 19772. ChatGPT literally has no idea there are letters involved.

(There are ways around this problem, which is why some AI/LLMs can figure it out.)

I had a fun moment where my standard paid ChatGPT 4o didn’t even know that ChatGPT 5 exists and was just released. chatgpt.com/share/68956d...

I can see why replacing your customer service and advertising teams with AI is a good idea

I asked an AI knower about it if you’re curious

My wife, who is a data scientist, just gave me a very similar version of that talk!

It's like an aphasic patient "explaining"