I have spent too much time arguing with ChatGPT already: it always misrepresents the subjects it is speaking about. In my opinion it is like Lem's demon: https://empathy.guru/2016/05/28/maxwells-demonsthe-pirate-pugg-and-the-true-nature-of-the-internet/, which spouts true random facts which are worthless. In this case it is even worse, because they are fabricated lies but which look authentic. The true value is in fact-checking the thing as you say.
A more accurate term for the LLM behaviour would be "confabulation", not "hallucination".
I'm tilting against windmills here - the talking heads have decided to call the LLM behaviour "hallucination". But it's not hallucination, which implies all kinds of things which are not true about the supposed hallucinators.
I half suspect the term "hallucination" came from people who wanted the ill-informed to believe that LLMs were actually intelligent, even something a lot like people. But maybe it came from non-technical naifs who already at least half-believed the LLMs were actually intelligent in some real sense. Or maybe it came from people of limited vocabulary, who'd never heard of confabulation. Or maybe it started as a metaphor among technical people, but now has new life as a term-of-art commonly misunderstood by outsiders.
Whatever the reason, the term "hallucinate" is misleading and inappropriate.
Other than that, I hope you are right that people will respond to the behaviour of LLMs by becoming less credulous, but I don't share your optimism. People believe whatever their in-group believes, or whatever makes them feel good, or whatever leaders they (foolishly) trust proclaim as truth. We have AFAICT decreasing ability to fact check those proclamations, compared to in my mostly pre-digital youth. So why not believe whatever implausible rubbish comes your way? Even if you discard theories like politician-you-hate is a Grey Alien and inconvenient-well-attested-bit-of-history is false, you still won't know whether any particular news report is true.
I have spent too much time arguing with ChatGPT already: it always misrepresents the subjects it is speaking about. In my opinion it is like Lem's demon: https://empathy.guru/2016/05/28/maxwells-demonsthe-pirate-pugg-and-the-true-nature-of-the-internet/, which spouts true random facts which are worthless. In this case it is even worse, because they are fabricated lies but which look authentic. The true value is in fact-checking the thing as you say.
I agree! I've found ChatGPT under-whelming as a research tool. The speed and fluency are amazing, but it tends to lack substance and credibility.
A more accurate term for the LLM behaviour would be "confabulation", not "hallucination".
I'm tilting against windmills here - the talking heads have decided to call the LLM behaviour "hallucination". But it's not hallucination, which implies all kinds of things which are not true about the supposed hallucinators.
I half suspect the term "hallucination" came from people who wanted the ill-informed to believe that LLMs were actually intelligent, even something a lot like people. But maybe it came from non-technical naifs who already at least half-believed the LLMs were actually intelligent in some real sense. Or maybe it came from people of limited vocabulary, who'd never heard of confabulation. Or maybe it started as a metaphor among technical people, but now has new life as a term-of-art commonly misunderstood by outsiders.
Whatever the reason, the term "hallucinate" is misleading and inappropriate.
Other than that, I hope you are right that people will respond to the behaviour of LLMs by becoming less credulous, but I don't share your optimism. People believe whatever their in-group believes, or whatever makes them feel good, or whatever leaders they (foolishly) trust proclaim as truth. We have AFAICT decreasing ability to fact check those proclamations, compared to in my mostly pre-digital youth. So why not believe whatever implausible rubbish comes your way? Even if you discard theories like politician-you-hate is a Grey Alien and inconvenient-well-attested-bit-of-history is false, you still won't know whether any particular news report is true.
Thanks for the feedback!