精读预计 3 分钟

How bigger data diminish the value of research to business

摘要

芝加哥大学访问教授 John Hand 对过去 30 年顶级会计期刊的研究显示，学术成果对现实商业的解释力下降了近 50 倍。研究指出，随着数据集规模增长 60 倍，研究者倾向于利用海量数据挖掘统计显著但经济意义微小的结论（即“过度捕捞”现象），导致研究脱离宏观经济图景。作者建议期刊应强制要求报告解释力，并鼓励通过手动收集数据来促使研究者进行更深入的思考。

荐读理由

在评估 AI 研究或市场调研时，你可以利用文中对“统计显著性”与“实际解释力”脱钩的论证，识别并过滤掉那些因大数据集产生的、但在实际业务决策中几乎无价值的微小结论（over-krilling）。

原文

How Bigger Data Diminish the Value of Research to Business

The explanatory power of accounting research findings has declined enormously over the past 30 years.

Most academic research claims a link to the world outside the ivory tower. Biology research may advance medical science; philosophers may offer insight into the human condition; and research by accounting professors can put new tools, methods or insights, and findings into hands of accountants, auditors, CFOs, and a wide variety of other businesspeople.

However, this last connection appears to be fraying. University of North Carolina’s John Hand, a visiting professor at Chicago Booth since 2017, examined a random selection of publications in the accounting field’s top research journals over the past 30 years and finds a large and steady weakening of the explanatory power of what researchers have historically focused on, and continue to.

“The real world cares about stuff that matters with a capital M, or at least a medium-sized M, yet we in accounting seem to be documenting things with tiny, tiny, tiny Ms,” he says.

Hand’s analysis included more than 200 articles published between 1995 and 2024 in the four top academic accounting journals. He identified the key independent variable in each paper along with the paper’s t-statistic. In the world of academic publishing, the t-stat denotes how well a chosen variable explains the effect that is being studied; t-stats over a certain threshold are considered significant.

However, explanatory power of the t-statistic is typically closely related in an inverse manner to the number of observations in a study’s dataset. When the number of observations used in accounting research papers grows larger (as it has in the past three decades, increasing 60-fold), it becomes easier to sift through data and uncover findings that are statistically significant based on the t-stat. However, those findings are tiny in terms of their explanatory power, and Hand argues that this is what matters most to the business world. The explanatory power of key independent variables in recent accounting papers is nearly 50 times smaller than in studies conducted in the mid-1990s, he finds.

The problem, Hand suggests, is that growing datasets send researchers hunting for minutiae—what he terms “over-krilling,” or homing in on the microscopic “krill” in the ocean of ideas. “People are losing sight of the big economic picture when they focus on these tiny results,” he says. In prior decades, Hand posits, when data were not as abundant and cheap to obtain, researchers had to be more deliberate and thoughtful about the kinds of questions they asked and the data they gathered. In doing so, they tended to focus on factors that were likely to be important to businesspeople.

Hand finds that more recent papers with less powerful findings are also more heavily footnoted and contain lengthier appendixes. While recognizing that these metrics are “imperfect proxies” for the tendency to over-krill, he argues that researchers who dive into the weeds spend more time carefully annotating their work, potentially as a means of exaggerating the perceived importance of their findings.

He proposes a list of recommendations to correct what he sees as a movement in the wrong direction. It includes straightforward measures, such as having journals require that the explanatory power of each key independent variable be reported, along with more substantial fixes. These include encouraging researchers not to simply use larger, digitally collected datasets but rather to collect data manually. This slower process would help the researcher think through the data being collected and what analyses, economic or otherwise, could prove informative and meaningful to business.

“If the work we do continues to decrease in explanatory power such that our scholarly findings in 30 years’ time explain just one thousandth of 1 percent of what’s happening to the outcomes that people care about, we risk becoming wholly irrelevant to business,” he says. “This isn’t a smart direction for us to be heading.”

Works Cited

John Hand, “Bigger Data + Tinier Results = The Wrong Direction,” Working paper, February 2026.

How bigger data diminish the value of research to business

How Bigger Data Diminish the Value of Research to Business

Related Topics

Related Topics