You know how Google’s new feature called AI Overviews is prone to spitting out wildly incorrect answers to search queries? In one instance, AI Overviews told a user to use glue on pizza to make sure the cheese won’t slide off (pssst…please don’t do this.)
Well, according to an interview at The Vergewith Google CEO Sundar Pichai published earlier this week, just before criticism of the outputs really took off, these “hallucinations” are an “inherent feature” of AI large language models (LLM), which is what drives AI Overviews, and this feature “is still an unsolved problem.”
No I still agree that we are far from LLMs being ‘thinking’ enough to be anywhere near this. But if we had a bunch of models similar to LLMs that could actually think, or if we really needed to select a person, I do think it would be possible to evaluate a bunch of the models/people to determine which ones are good at distinguishing fake information.
All I’m saying is I don’t think the limitation is actually our ability to select for capability in distinguishing fake information, I think the only limitation is fundamental to how current LLMs work.
Yes, my point wasn’t that it could never be achieved but that LLMs are in a completely different category, which we agree on I think. I was comparing them to humans who have trouble with critical thinking but can easily spot AI’s hallucinations to illustrate the vast gulf.
In both cases I think there are almost certainly more barriers in the way than an education. The quest for a truthful AI will be as contentious as the quest for truth in humans, meaning all the same claim-counterclaim culture-war propaganda tug of war will happen, which I think is the main reason for people being miseducated against critical thinking. In a vacuum it might be a simple technical and educational challenge, but the reason this is a problem in the first place is that we don’t exist in a political vacuum.