AI advice made people 3x less accurate but 2x confident, researchers found
摘要
三所法意大学研究人员发现，提供 AI 建议后，人们说‘我不知道’的比例从 44% 降至 3%，准确率从 27% 降至 9%，自信从 30% 升至 76%。研究故意选用 AI 常错的电影视觉细节问题，使用 Step 3.5 Flash 模型，部分原本答对者因 AI 而变错。货币激励仅将‘我不知道’升至 8%、准确率升至 16%，仍远低于无 AI 基线。研究指出 AI 产品设计从不说‘我不知道’，可能让使用者尤其是儿童减少认知界限的识别。
荐读理由
研究用AI常答错的视觉细节题测试，发现AI建议把‘我不知道’比例从44%压到3%、准确率从27%降到9%，信心却从30%升到76%，这为你提供数据支撑的反共识：AI集成可能导致用户‘认知投降’，在产品设计中需预判并缓解过度依赖风险。
原文
Image by: Canva
TL;DR
Researchers found AI advice suppressed judgment suspension from 44% to 3%, accuracy from 27% to 9%, while confidence rose from 30% to 76%. People trusted wrong AI answers.
Researchers from three French and Italian universities found that access to AI advice collapsed people’s willingness to say “I don’t know” from 44% to 3%. Accuracy dropped from 27% to 9%. Confidence, meanwhile, rose from 30% to 76%. “People became much worse, the accuracy was only one third, but they were twice as confident,” said Valerio Capraro, associate professor at the University of Milano-Bicocca.
The study, authored by Capraro with Chiara Marcoccia of École Normale Supérieure and Walter Quattrociocchi of Sapienza University of Rome, deliberately used questions where AI models typically fail: visual details from films, such as the colour of a team’s uniform in Bend It Like Beckham. The researchers used Step 3.5 Flash, a model that was usually wrong on these questions, precisely so any reduction in judgment could not be explained as sensible delegation to a reliable tool. Some participants who would have answered correctly on their own asked the AI and became wrong.
Monetary incentives helped, but not much. Willingness to admit ignorance rose from 3% to 8% and accuracy from 9% to 16%, both still well below the no-AI baselines of 44% and 27%. Wharton researchers coined the term “cognitive surrender” earlier this year to describe the same phenomenon: people accepting incorrect AI answers 80% of the time while reporting higher confidence than those working without AI. The new study adds a sharper data point. It is not just that people trust wrong AI answers. It is that the mere availability of AI suppresses the cognitive habit of recognising what you do not know.
The 💜 of EU tech
The latest rumblings from the EU tech scene, a story from our wise ol' founder Boris, and some questionable AI art. It's free, every week, in your inbox. Sign up now!
“For humans, the capacity to say ‘I don’t know’ is very important because it represents the recognition of the limits of our own knowledge,” Capraro said. He is particularly concerned about children, who are growing up with these systems before they have developed critical thinking skills. Google’s AI search overhaul replaced links with confident AI-generated summaries, and Common Sense Media this week called that design an “unacceptable risk” for students. The pattern is consistent: AI products are designed to answer, never to say “I don’t know.” The humans using them are learning to do the same.
Get the TNW newsletter
Get the most important tech news in your inbox each week.
这条对你有帮助吗？