To understand AI hallucinations one must first understand two philosophical ideas: 1) Plato's "man in a cave" 2) Garbage in, garbage out To the machine without eyes or ears, text is their only window to the world. Without any context outside of that, how can it tell the truth from fiction?
If your only sense was the ability to read, how could one tell the difference between the actual events of the Titanic disaster and the screenplay of the movie of the same name? With zero other senses or inputs, both can seem equally valid. Confusing reality & fiction become unavailable.
If I have a science question, I always tell it "Source from peer reviewed scientific journals & papers only." If I want news I say "Ignore anything from Fox News." So yes, you can train it to ignore bullshit that causes hallucinations, but you *do* have to train it. It won't know better otherwise.